-- Logs begin at Tue 2022-10-25 12:44:40 UTC, end at Tue 2022-10-25 13:52:08 UTC. -- Oct 25 13:10:32.294202 dv-cinder-862432-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 25 13:10:32.301868 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 25 13:10:32 2022] *** Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: nodename: dv-cinder-862432-2-iscsi Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: machine: x86_64 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: clock source: unix Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: pcre jit disabled Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: detected number of CPU cores: 4 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: current working directory: / Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: detected binary path: /usr/bin/uwsgi-core Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: your processes number limit is 47680 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: your memory page size is 4096 bytes Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: detected max file descriptor number: 2048 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: lock engine: pthread robust mutexes Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: thunder lock: enabled Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 25 13:10:32.306231 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 25 13:10:32.344405 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: Python main interpreter initialized at 0x55d4293719c0 Oct 25 13:10:32.344405 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: python threads support enabled Oct 25 13:10:32.344405 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: your server socket listen backlog is limited to 100 connections Oct 25 13:10:32.344405 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: your mercy for graceful operations on workers is 90 seconds Oct 25 13:10:32.344779 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: mapped 403077 bytes (393 KB) for 2 cores Oct 25 13:10:32.344779 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: *** Operational MODE: preforking *** Oct 25 13:10:32.345097 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: *** uWSGI is running in multiple interpreter mode *** Oct 25 13:10:32.345097 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: spawned uWSGI master process (pid: 81424) Oct 25 13:10:32.345322 dv-cinder-862432-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 25 13:10:32.346196 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: spawned uWSGI worker 1 (pid: 81436, cores: 1) Oct 25 13:10:32.346196 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: spawned uWSGI worker 2 (pid: 81437, cores: 1) Oct 25 13:10:32.346355 dv-cinder-862432-2-iscsi devstack@keystone.service[81424]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 25 13:10:36.571569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 25 13:10:36.571569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Oct 25 13:10:36.572386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 25 13:10:36.572386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Oct 25 13:10:36.572803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 25 13:10:36.572803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: from cryptography.utils import int_from_bytes Oct 25 13:10:36.572912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 25 13:10:36.572912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: from cryptography.utils import int_from_bytes Oct 25 13:10:36.621775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 25 13:10:36.621979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 25 13:10:36.622073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 25 13:10:36.622151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 25 13:10:36.622216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] command line args: [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 25 13:10:36.622281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] command line args: [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 25 13:10:36.622349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 25 13:10:36.622557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 25 13:10:36.622647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ================================================================================ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 25 13:10:36.622715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ================================================================================ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 25 13:10:36.622786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.622960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_dir = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_dir = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] config_source = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] debug = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] config_source = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.623784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] debug = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.625183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.626982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_config_append = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_config_append = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_dir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.627944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_dir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.628997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.629943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] publish_errors = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] publish_errors = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.630922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] standard_threads = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] standard_threads = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_journal = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.631996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_json = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_journal = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_stderr = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_json = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_stderr = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] use_syslog = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] use_syslog = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 25 13:10:36.632823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.632953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.633917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.634905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.635937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.636899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.637957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.638931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.639879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.640983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.641933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.642930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.643870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.644908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.645962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.646988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.647928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.648912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.649949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.650949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.651894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.652894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.653974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.654897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.655921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.656919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.external = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.external = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.657925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.password = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.password = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] auth.token = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.658939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] auth.token = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.659975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.660997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.661908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.662928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.663969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.664978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.665999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.666928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.667910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.668926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.669929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.670950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.671952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672846 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.672957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.673944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.674956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.675993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.676993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.677992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.678887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.679927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.680938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.681947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.682961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.683932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.684926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.685937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.686932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.687983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688453 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.688901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.689940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.690957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.691892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.692900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.693959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.694926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.695891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.696930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.697940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.698964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.699924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.700946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701768 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.701882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.702947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.703968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.704928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.705965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 25 13:10:36.706874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.706984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 25 13:10:36.707133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 25 13:10:36.736325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.736576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.736665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.736789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.736871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.737032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.745183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.745371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.774739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.774897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.775110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.775258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.826755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.826982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.875418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.875647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.876390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:36.876649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.035493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.035703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.035829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.035911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.035998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.036632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.059601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.059821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.059991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.060961 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.061170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.072020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.072362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.085402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.085536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.085806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.085942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81437) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.086877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.087091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81436) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 25 13:10:37.103736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.103947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 25 13:10:37.104208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.104332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.104457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 25 13:10:37.104559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 25 13:10:37.104703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.104788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.104932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 25 13:10:37.105053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.105185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.105454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.105575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.105710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.105968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.106233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.115140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.115446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.115661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.115785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.115920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.116018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.116177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.116486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.122899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.125806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.126091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.126371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.126492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.126703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.126828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.126983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.127939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.128972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.129250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.151056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.151622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.152539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.152822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.153095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.153379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.159164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.159457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.159737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.159859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.160014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.160317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.166255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.166546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.166772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.167062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.170676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.170968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.171211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.171309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.171543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.171687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.171823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.171901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.172895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.173188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.188163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.188441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.188747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.189066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.192382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.192946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.195591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.195885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.196188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.196486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.200106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.200719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.203529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.203869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.207093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.207417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.207540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.207678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.207758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.207997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.208867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.224408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.224612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.224709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.224923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.225045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.225132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.225208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.225438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.232014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.232309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.233566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.233861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.235111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.235425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.235547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.235805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.238154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.238438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.238560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.238771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.238894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.239065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.239254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.239403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.239545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.239756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.250440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.250729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.250851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.251090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.251240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.251394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.251562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.251770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.251934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.252022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.252110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.252297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.252436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.252660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.267636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.268049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.270511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.270958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.273471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.273925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.274889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.275380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.276520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.276818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.276976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.277070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.277294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.277577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.283474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.284000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.284955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.285469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.287982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.288271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.288493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.288616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.288741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.288837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.289052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.289331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.296608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.296896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.297136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.297289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.297406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.297718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.305204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81437) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.305485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.305623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81436) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 25 13:10:37.305751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.305836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306146 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.306894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.307925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81437) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.308725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.309000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81436) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 25 13:10:37.341952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d4293719c0 pid: 81437 (default app) Oct 25 13:10:37.342294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d4293719c0 pid: 81436 (default app) Oct 25 13:10:37.345215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cef6c-f24f-41c6-a5df-62d515c06f51 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:37.345558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cef6c-f24f-41c6-a5df-62d515c06f51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:37.345846 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cef6c-f24f-41c6-a5df-62d515c06f51 None None] PATH_INFO: `/v3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:37.350020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Tue Oct 25 13:10:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:44.139594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:44.140441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:44.140441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:44.147636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:44.155721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:44.156262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:44.156262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:44.226937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: INFO dbcounter [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] Registered counter for database keystone Oct 25 13:10:44.248494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG oslo_db.sqlalchemy.engines [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 25 13:10:44.256983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writer thread running {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 25 13:10:44.434077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81437) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 25 13:10:44.434355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG passlib.handlers.bcrypt [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81437) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 25 13:10:44.493973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:44.496681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:44.643379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 25 13:10:45.110522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:45.182855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: INFO dbcounter [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] Registered counter for database keystone Oct 25 13:10:45.207406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG oslo_db.sqlalchemy.engines [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 25 13:10:45.218826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writer thread running {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 25 13:10:45.341748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273323c-0ea3-4797-8e2d-6ab4f3ea3e65 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:45.346778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:45.347431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:45.347704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:45.348075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:45.356807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:45.407992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 25 13:10:45.408302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a42a1-7e87-4703-b32d-aa93c8fae60d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:45.418259 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Tue Oct 25 13:10:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:10:45.424618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:45.443006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5e12c-c6d2-4ff1-826f-e741f3f06918 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:45.448592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:45.449707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:45.449985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:45.450252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] PATH_INFO: `/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:45.454200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] RBAC: Authorizing `identity:get_project(project_id=e7cf0d7d46da4af3b19a1ac43ed2227b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:45.504301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING keystone.common.rbac_enforcer.enforcer [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 25 13:10:45.504645 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721606e0-5fe2-4481-9ae7-0926dd450a07 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:45.509168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:10:45 2022] GET /identity/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:47.606331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de691c9-c35d-4245-87d3-44563e3c0a45 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:47.606936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de691c9-c35d-4245-87d3-44563e3c0a45 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:47.606936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de691c9-c35d-4245-87d3-44563e3c0a45 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:47.611086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:10:47.617709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:47.618435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:47.618579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:47.652048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:47.653602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:47.662179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:10:47.927025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de691c9-c35d-4245-87d3-44563e3c0a45 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:47.946783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de691c9-c35d-4245-87d3-44563e3c0a45 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:47.952864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:47.953477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:47.953761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:47.954042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:47.963186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:47.965011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139afb6-b3ab-47b7-bcb1-cbb37c63e455 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:47.974321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Tue Oct 25 13:10:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:10:47.982566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:47.997139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b66bf-e183-4a2a-8d84-8f074dc42c59 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:48.004231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:48.004855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:48.005133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:48.005397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] PATH_INFO: `/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:48.006864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] RBAC: Authorizing `identity:get_user(user_id=ec20b39fa45c4429b8c7ce4fb9b19e90)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:48.009008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ecdda-bfda-49f4-b47c-bf11ae5415fe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:48.010540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Tue Oct 25 13:10:47 2022] GET /identity/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90 => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:50.193316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6ffc-6af4-4cce-a180-119800234267 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.193895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6ffc-6af4-4cce-a180-119800234267 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.193895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6ffc-6af4-4cce-a180-119800234267 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.195205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:50.200692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.201119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.201270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.216602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d90d7-d8a9-450d-847c-f57f8e97a49b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.216887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d90d7-d8a9-450d-847c-f57f8e97a49b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.217195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d90d7-d8a9-450d-847c-f57f8e97a49b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.218567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:10:50.221919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:50.223642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:50.223796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.224029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.224304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.265667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81436) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 25 13:10:50.266091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG passlib.handlers.bcrypt [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81436) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 25 13:10:50.308026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:10:50.314436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:50.316075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:50.416635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:10:50.571853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:50.597086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:50.678054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:50.680506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d370e16-5a0d-47c6-8f42-51de292421d5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:50.716804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:50.717452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.717731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.718008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.730122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:10:50.739172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:10:50.740189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:10:50.741020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:10:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:10:50.746185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:50.780226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae653a-5897-43a4-bb1e-e9e96ad69555 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:50.782571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:50.818652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:50.819555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.819836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.820106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.821082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:50.822572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:50.829296 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:50.834424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:10:50.834424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:10:50.834424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [parameters: {'id': '146285aaac0244aa91d3aa58ebf5cb20', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 25 13:10:50.834424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 25 13:10:50.848375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02371-7f56-4399-bac4-0a5e47c1bd5c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:10:50.853508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 25 13:10:50.854457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 25 13:10:50.855409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Oct 25 13:10:50.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 25 13:10:50.857135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:10:50.857879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': '146285aaac0244aa91d3aa58ebf5cb20', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 25 13:10:50.858679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:10:50.860334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:50.861404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 8/15] 10.222.0.36 () {60 vars in 1137 bytes} [Tue Oct 25 13:10:50 2022] POST /identity/v3/domains => generated 190 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:10:50.865163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:50.883324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:50.883934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:50.884223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:50.884502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:50.885162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:10:50.885463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:50.886867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:50.898586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 8/16] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:10:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:50.904247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:50.966842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2d664-dbe8-4142-8571-3fc0329496f6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:51.000619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:51.001550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:51.001830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:51.002110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:51.004816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e18cd-42f0-4526-ba42-422f9a43956e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:51.021128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:51.022659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:51.039430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:51.040068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:51.040346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:10:51.040598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:10:51.041470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:10:51.042305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:10:51.042305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:10:51.042305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:10:51.042478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] PATH_INFO: `/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:51.042544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:10:51.042606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] RBAC: Authorizing `identity:get_user(user_id=ec20b39fa45c4429b8c7ce4fb9b19e90)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:51.043657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:51.045355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:10:50 2022] GET /identity/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:51.047578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:51.050569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:51.065672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:51.073166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:51.150932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674d99d-f74f-48e8-a534-70d4f9762489 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:51.152365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052057f-640c-4cdc-a15f-8fb24900765e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:51.185033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:51.185197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:51.185675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:51.185803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:51.185926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:51.186057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:51.186196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:51.186292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:51.186848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:10:51.187175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:51.187353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:51.188605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bc657-e81c-466c-b5cd-9ee447566466 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:51.188790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:51.190553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:51.196210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:51.207177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:51.280634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:51.307139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0ee6-f63a-42e8-a37b-b5fb2846b400 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:51.344885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:51.345469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:51.345747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:51.346032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] PATH_INFO: `/v3/domains/default/users/ec20b39fa45c4429b8c7ce4fb9b19e90/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:51.348233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ec20b39fa45c4429b8c7ce4fb9b19e90, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:51.349730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8bfae-efb5-4a74-a864-edd4a0a9d398 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:51.465813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 11/21] 10.222.0.36 () {58 vars in 1361 bytes} [Tue Oct 25 13:10:51 2022] PUT /identity/v3/domains/default/users/ec20b39fa45c4429b8c7ce4fb9b19e90/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:10:53.835207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7962a6-6e94-4982-8644-47189dfe9e6e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:53.835743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7962a6-6e94-4982-8644-47189dfe9e6e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:53.835743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7962a6-6e94-4982-8644-47189dfe9e6e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:53.837110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:53.843313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310ffee-797b-42e9-8073-08aa8cf407c8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:53.843777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310ffee-797b-42e9-8073-08aa8cf407c8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:53.843871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310ffee-797b-42e9-8073-08aa8cf407c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:53.871793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6310ffee-797b-42e9-8073-08aa8cf407c8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:53.873334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6310ffee-797b-42e9-8073-08aa8cf407c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:53.915910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce87f1-5283-4c75-bc34-b986ed03ffbe None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:53.916086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce87f1-5283-4c75-bc34-b986ed03ffbe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:53.916362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce87f1-5283-4c75-bc34-b986ed03ffbe None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:53.917849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 12/23] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:10:53.931112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797030c0-f08a-4190-88a6-fe9df2494da8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:53.931424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797030c0-f08a-4190-88a6-fe9df2494da8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:53.931702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797030c0-f08a-4190-88a6-fe9df2494da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:53.956168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-797030c0-f08a-4190-88a6-fe9df2494da8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:53.957386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-797030c0-f08a-4190-88a6-fe9df2494da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:54.005471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:10:54.093151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:10:54.228541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a414be-1986-4af6-a192-4d85e34dc366 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.228833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a414be-1986-4af6-a192-4d85e34dc366 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.229019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a414be-1986-4af6-a192-4d85e34dc366 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.233256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:54.239068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.239388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.239666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.264110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:54.265300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:54.268939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:54.348392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:10:54.498774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a414be-1986-4af6-a192-4d85e34dc366 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:54.552467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:54.585898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:54.647448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a414be-1986-4af6-a192-4d85e34dc366 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:54.696001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20c0d3-0046-498d-a902-2674cf83c411 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:54.715976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:54.716632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.716914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.717186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.730301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:54.732055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-98132fec-4724-4bb2-bd17-39e0c8c23658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:10:54.751273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:10:54.752178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 14/28] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c9a9f-4245-40f2-b933-52742718b4ea None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.753026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c9a9f-4245-40f2-b933-52742718b4ea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.754042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c9a9f-4245-40f2-b933-52742718b4ea None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.755655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 15/29] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:54.756586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c9a9f-4245-40f2-b933-52742718b4ea None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:54.773225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:54.773870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.774145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.774558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.779623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:54.781062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445a21a-8c76-4535-bc1d-55f8b2b1f7a7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:54.803962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:54.879858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c9a9f-4245-40f2-b933-52742718b4ea None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:54.904248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:54.918222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:54.919274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.919750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.920212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.921082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:10:54.921618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:54.923343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:54.924049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 15/30] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Oct 25 13:10:54 2022] POST /identity/v3/roles => generated 219 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:10:54.925591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:54.925876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:54.926305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:54.945126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 16/31] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:54.946384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:54.959076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:54.960898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:55.056560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 16/32] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:10:55.057716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:55.066585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177863a5-16f5-4a34-b1fc-010a2d9bdb69 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:55.105231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:55.106334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.106812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.107370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.108274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:55.109877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f3b75-c4a6-40cc-ae5e-36b0acfeca84 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:55.144746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 17/33] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Oct 25 13:10:54 2022] POST /identity/v3/roles => generated 225 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:10:55.146381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31683c4-61ef-4665-8a07-c9efdbe60eee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.146869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31683c4-61ef-4665-8a07-c9efdbe60eee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.147412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31683c4-61ef-4665-8a07-c9efdbe60eee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.149323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 18/34] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:55.155415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.156034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.156508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.176336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:55.177956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:55.179232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d428daa8-589c-4fae-bb8e-030e464b35d7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:55.217037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:55.224345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:55.226336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:55.227644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.229121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.229591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.230498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:55.232575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:55.267674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 19/35] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:10:55.269042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 17/36] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:10:55 2022] POST /identity/v3/projects => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:10:55.479603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:55.590741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcb6f1-416d-44a6-b88a-24dc9c3893db None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:55.630288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:55.630944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.631273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.631682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.632622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:55.643386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:10:55.650042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:10:55.650851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:10:55.651945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 20/37] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:10:55.657330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:55.694010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:55.751201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6957bf-ebe2-4e8f-9c3b-d95a850e9a96 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:55.769101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87500b-d384-4b70-9762-e621e8dbd335 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:55.786848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:55.787514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.787790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.788061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.788745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:55.790144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb95f1-d9b2-4131-9b12-f2e838ca8c19 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:55.805608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 18/38] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Oct 25 13:10:55 2022] POST /identity/v3/roles => generated 223 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:10:55.805957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:55.806690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.806861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8a1f-35ef-4330-b412-46b48bf8a69f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.807008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.807112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8a1f-35ef-4330-b412-46b48bf8a69f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.807281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.807443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8a1f-35ef-4330-b412-46b48bf8a69f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.808032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:10:55.808344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:55.808798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 19/39] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:55.810362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:55.815129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:55.815813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:55.816296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:55.820332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 21/40] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:55.826907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:55.838787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:55.840423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:55.888305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:55.929831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 20/41] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:10:56.041439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:56.049195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029c93-39ed-4f21-a0e1-96451c368544 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:56.083855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:56.084497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:56.084781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:56.085056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] PATH_INFO: `/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:56.086208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] RBAC: Authorizing `identity:get_user(user_id=ec20b39fa45c4429b8c7ce4fb9b19e90)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:56.087698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:56.089319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 22/42] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:10:55 2022] GET /identity/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:56.095311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:56.194749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923318d-db28-4b76-9c3d-9c30c6241c69 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:56.200106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:56.228360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:56.228996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:56.229271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:56.229563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:56.230661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:56.232251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:56.233986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 21/43] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:56.239714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:56.280840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:56.301643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e503b-fc72-406a-b389-96eeff41d803 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:56.335670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:56.336304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:56.336586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:56.336857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:56.337959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:56.339438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:56.340311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22603c6-ac89-4d2d-bd93-10180187d9f3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:56.341035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 23/44] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:56.346690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:56.377097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:56.377535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:56.377722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:56.378022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:56.378298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:56.379379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.domain.id=default, user.id=ec20b39fa45c4429b8c7ce4fb9b19e90) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:10:56.379810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:56.381275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bda3a-4d82-417e-8039-e39b27e521b6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:56.389836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 22/45] 10.222.0.36 () {56 vars in 1322 bytes} [Tue Oct 25 13:10:56 2022] GET /identity/v3/role_assignments?user.id=ec20b39fa45c4429b8c7ce4fb9b19e90&scope.domain.id=default&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:10:56.450584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0826-bb5d-4ea2-bcd3-858e3a43d456 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:56.488251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:56.488878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:56.489160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:56.489428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:56.490095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:56.491605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619564e4-40ef-48e1-89ff-f95c6473747e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:56.525899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:10:56 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:10:58.672017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57628aa-23d9-4be6-9358-f7f162bb1050 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:58.672476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57628aa-23d9-4be6-9358-f7f162bb1050 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:58.672573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57628aa-23d9-4be6-9358-f7f162bb1050 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:58.673996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 23/47] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:10:58.680518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:58.681224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:58.681420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:58.709128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:10:58.710331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:10:58.781783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 25/48] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:10:59.052960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57628aa-23d9-4be6-9358-f7f162bb1050 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:59.162143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57628aa-23d9-4be6-9358-f7f162bb1050 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:59.164522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:59.206301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:59.206924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:59.207202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:59.207506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:59.208689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:59.210149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6696812-675f-46dc-a6b7-460b6e85adf0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:59.211735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 24/49] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:10:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:10:59.217535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:10:59.232015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:10:59.321227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dfab5-06c9-47a2-b828-bd46e7bd3080 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:10:59.357858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:10:59.358482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:10:59.358770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:10:59.359045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:10:59.359867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:10:59.361386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bc29c-1012-4ed5-8e24-90553e0c69af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:10:59.393429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 26/50] 10.222.0.36 () {60 vars in 1140 bytes} [Tue Oct 25 13:10:59 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:01.496426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7967-4eb1-423c-9541-1603f583b9e1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:01.496964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7967-4eb1-423c-9541-1603f583b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:01.496964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa7967-4eb1-423c-9541-1603f583b9e1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:01.499122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 25/51] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:01.506298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:01.506673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:01.506927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:01.535880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:01.537085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:01.598803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:01.611498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 27/52] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:01.877886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa7967-4eb1-423c-9541-1603f583b9e1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:02.006012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa7967-4eb1-423c-9541-1603f583b9e1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:02.040823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:02.041476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:02.041753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:02.042032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:02.043190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:02.044702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36c1bc-9bf4-43ab-848f-0224dd22cdef None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:02.046258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:02.052118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:02.157912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1367f-68ce-4a63-8834-81ac760365e1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:02.192225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:02.192832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:02.193109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:02.193380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:02.194034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:02.195508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87560-df18-401e-a4bc-3cd30f0712be None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:02.225627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 28/54] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:02 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:05.314878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c403a6-1f41-4f74-a9a1-02ef30c29047 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.315456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c403a6-1f41-4f74-a9a1-02ef30c29047 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.315456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c403a6-1f41-4f74-a9a1-02ef30c29047 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.316850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 27/55] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:05.322233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547250-4893-4fa9-ac8e-7c715e926aa1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.322608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547250-4893-4fa9-ac8e-7c715e926aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.322796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547250-4893-4fa9-ac8e-7c715e926aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.348102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eb547250-4893-4fa9-ac8e-7c715e926aa1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:05.349401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eb547250-4893-4fa9-ac8e-7c715e926aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:05.409870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:05.440697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 29/56] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:05.682410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779879-f17d-46ae-abd2-643a3df08d1f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.683052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779879-f17d-46ae-abd2-643a3df08d1f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.683557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f779879-f17d-46ae-abd2-643a3df08d1f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.685119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 28/57] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:05.690606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6c46-0100-43e3-a691-edb918eb3ccc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.691308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6c46-0100-43e3-a691-edb918eb3ccc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.691785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a6c46-0100-43e3-a691-edb918eb3ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.720255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-597a6c46-0100-43e3-a691-edb918eb3ccc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:05.721789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-597a6c46-0100-43e3-a691-edb918eb3ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:05.757368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f779879-f17d-46ae-abd2-643a3df08d1f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:05.810319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:05.814583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 30/58] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:05.819372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad171-080c-4cca-97fc-19f4a5f51090 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.819694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad171-080c-4cca-97fc-19f4a5f51090 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.819970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad171-080c-4cca-97fc-19f4a5f51090 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.821283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 31/59] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:05.838219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7750-e801-4f35-90f0-8cf1e00a0eda None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.838498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7750-e801-4f35-90f0-8cf1e00a0eda None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.838771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7750-e801-4f35-90f0-8cf1e00a0eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.864249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ce7e7750-e801-4f35-90f0-8cf1e00a0eda None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:05.865438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ce7e7750-e801-4f35-90f0-8cf1e00a0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:05.906350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f779879-f17d-46ae-abd2-643a3df08d1f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:05.943409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:05.944663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:05.945136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:05.945613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:05.959942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:05.966973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:05.967954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:05.968748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:05.971520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 29/60] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:05.980059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.009028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 32/61] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 25 13:11:06.077175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a680c-e810-41d6-b4cf-ddf18e2fd1aa None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.077796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a680c-e810-41d6-b4cf-ddf18e2fd1aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.078286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a680c-e810-41d6-b4cf-ddf18e2fd1aa None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.081314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 33/62] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:06.088341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.089087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.090050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.110943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:06.112799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:06.121880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.125525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64744-96ca-4c21-a622-417dad6b9c80 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.167748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.180164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.180803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.181192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.181464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.182186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:06.182581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:06.184133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:06.200678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 30/63] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:06.201779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.211656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 34/64] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:06.213032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.316671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ea80e-dd2c-4aa4-b2bc-f4098e5278a4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.340972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ecdd2-4e09-4df5-8577-2dc1f0a9ac90 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.380784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.381454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.381741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.382020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.397671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:06.399229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ea828-c0a1-433a-aa60-accb8da3cd57 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:06.402890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 35/65] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:06.408190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aeae6-8764-4866-916e-98b5b4dc5070 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.408190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aeae6-8764-4866-916e-98b5b4dc5070 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.408190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aeae6-8764-4866-916e-98b5b4dc5070 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.408190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 36/66] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:06.408190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aeae6-8764-4866-916e-98b5b4dc5070 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.465081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.477456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.478108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.478407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.478660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.489256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:06.494501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:06.495577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:06.496353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 31/67] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:06.496978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.521629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.559987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aeae6-8764-4866-916e-98b5b4dc5070 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.599976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.600645 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.600923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.601196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.606412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c097d-526c-48e2-8156-31a2116fd5ac None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.611623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-36cd56da-21f0-467d-a6f7-cc4f7acb9ba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:06.616515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:06.617434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:06.618191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 37/68] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:06.619466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.619739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.620002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.639617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:06.641845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:06.651878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.652602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.652977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.653322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.654506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:06.657056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:06.658723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 32/69] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:06.664454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.718016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 38/70] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:06.719464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.737855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.766116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.776436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff473b3d-1fbc-4a14-993c-e35b58462bf4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.810507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.811248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.811713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.812064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.812739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:06.813071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:06.814578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:06.821499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afc5ef-6ac1-4163-bc19-669c23abc8ce None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.827777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 33/71] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:06.828404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.856085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.856717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.856992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.857264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.867365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:06.872603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:06.873445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:06.874238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 39/72] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:06.879379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.934172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b0ede-ac4b-4fe5-9238-fe6151f35a19 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.961707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:06.968444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:06.969011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:06.969316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:06.969726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:06.970266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:06.970574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:06.972007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:06.984969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 34/73] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:06.984969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:06.989976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a848b-fd19-4f9e-974a-22b636bd51d4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:06.998461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.022997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.023695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.023979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.024254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/users/ad176173d8984450b1076196825032c1/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.026655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, user_id=ad176173d8984450b1076196825032c1, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.028151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.040705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 40/74] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:06 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/users/ad176173d8984450b1076196825032c1/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:07.041520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.089378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac34b4f-1f98-44c5-91fb-37a27eb12c7e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.127910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.128568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.128909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.129181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.130699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.132411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.139073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 35/75] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:06 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.139073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.165652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe98d8-b08b-4b0a-a42e-24e69b6cb8f9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.202474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.203401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.203724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.203998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.205040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:07.205345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.206785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.212872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.221975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 41/76] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.222330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.231489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.254938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a6d0-3b07-4d79-8c94-141bc91e98d9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.293819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.294959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.295351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.295915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] PATH_INFO: `/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.297079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] RBAC: Authorizing `identity:get_user(user_id=ec20b39fa45c4429b8c7ce4fb9b19e90)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.298573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.304693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 36/77] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.304693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.332877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f494f78-6462-413e-a6a9-817144653f7a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.366150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.366779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.367050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.367365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.368461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.369905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.371543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 42/78] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.375315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.424685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecef02-ea01-45f2-aaf3-eb459f3fdf53 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.442863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.463231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.463884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.464168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.464443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.465631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.467099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.468796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 37/79] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.469754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.483042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59e6fa-2fb3-436e-b4b1-73737e113e19 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.495087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.518273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.518911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.519288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.519565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.520704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.522154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.523851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 43/80] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.528402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.588139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362c06c-4cb3-43d6-b3ce-d7bad4728583 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.625428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef252c-03a2-44b7-b299-fed8cbcdd66e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.627276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.627931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.628207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.628479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.629931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.631452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.633762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 38/81] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.633886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.665266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.665885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.666168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.666442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.667100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.668610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.678415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.703196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 44/82] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:07 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:07.703196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.747335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:07.761413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd2e91-6f28-4227-ad48-ecbfaaf9ac5a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.802592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.803405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.803753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.804094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/users/ad176173d8984450b1076196825032c1/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.806495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, user_id=ad176173d8984450b1076196825032c1, role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.808779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.835250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 39/83] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:07 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/users/ad176173d8984450b1076196825032c1/roles/45f94767c7a04887a34b82d23eb15d91 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:07.836376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:07.860139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52250c03-6ad9-4cee-a878-0b838a22ce19 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:07.930330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:07.931019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:07.931378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:07.932360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] PATH_INFO: `/v3/projects/03689a2def334ebf8fda976106f097ad` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:07.934133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] RBAC: Authorizing `identity:get_project(project_id=03689a2def334ebf8fda976106f097ad)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:07.936018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:07.939035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 45/84] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:07 2022] GET /identity/v3/projects/03689a2def334ebf8fda976106f097ad => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:07.945097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:08.014877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb1002-0f40-4e70-8103-ce29f0b7fb55 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:08.036620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:08.064966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:08.065676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:08.065957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:08.066356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/users/ec20b39fa45c4429b8c7ce4fb9b19e90/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:08.068654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, user_id=ec20b39fa45c4429b8c7ce4fb9b19e90, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:08.070291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe727a2-f352-4408-b9a8-7326478b9102 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:08.088237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 40/85] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:07 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/users/ec20b39fa45c4429b8c7ce4fb9b19e90/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:08.095591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989456d6-1aee-457f-abc7-919729fd4437 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:08.138260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:08.139054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:08.139382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:08.139663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] PATH_INFO: `/v3/projects/03689a2def334ebf8fda976106f097ad/users/ad176173d8984450b1076196825032c1/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:08.142183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] RBAC: Authorizing `identity:create_grant(project_id=03689a2def334ebf8fda976106f097ad, user_id=ad176173d8984450b1076196825032c1, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:08.143834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a8696-1395-48e5-920b-e82858e66d6e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:08.160517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 46/86] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:07 2022] PUT /identity/v3/projects/03689a2def334ebf8fda976106f097ad/users/ad176173d8984450b1076196825032c1/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:09.423855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6048e-f499-4303-9189-249239e6456b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:09.424745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6048e-f499-4303-9189-249239e6456b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:09.425171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6048e-f499-4303-9189-249239e6456b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:09.426699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 41/87] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:09.432677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:09.433434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:09.433850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:09.457687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:09.459206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:09.529810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 47/88] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:10.055925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6048e-f499-4303-9189-249239e6456b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:10.066016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:10.178121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6048e-f499-4303-9189-249239e6456b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:10.221978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:10.222628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.222908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.223207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.240121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:10.245643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:10.246482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:10.247288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 42/89] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:10.253034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:10.320273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:10.381994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81182e-c6e0-4d97-8962-b5e46f0526a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:10.419927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:10.420551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.420832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.421111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.421789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:10.422094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:10.423679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:10.433454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 48/90] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:10.439491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:10.528405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:10.570043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f942dd-db6b-4b7a-a349-465ba0cd2001 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:10.610148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:10.610714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.610997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.616455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.618801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:10.620334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0db8a-2f16-4b99-89d8-84304aa4fef2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:10.623243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 43/91] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:10 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:10.629542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:10.709011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533174-c0be-47d6-8c4c-1d1b5cc1d2a8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.709358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533174-c0be-47d6-8c4c-1d1b5cc1d2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.709571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533174-c0be-47d6-8c4c-1d1b5cc1d2a8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.711081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 44/92] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:10.716728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bfc6-0b9b-4364-9d7a-c371cf2fcae8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.717535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bfc6-0b9b-4364-9d7a-c371cf2fcae8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.718289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bfc6-0b9b-4364-9d7a-c371cf2fcae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.740109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-57e3bfc6-0b9b-4364-9d7a-c371cf2fcae8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:10.741633 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-57e3bfc6-0b9b-4364-9d7a-c371cf2fcae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:10.854409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 45/93] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:10.858083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449009-9bdb-482a-84ea-c1919a0dec9c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:10.881875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:10.915029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:10.915683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.915970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.916248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.931291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9126-08f8-4617-b0f8-3ccedf9d54c1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.931581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9126-08f8-4617-b0f8-3ccedf9d54c1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.931849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9126-08f8-4617-b0f8-3ccedf9d54c1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.933433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 46/94] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:10.938699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:10.938983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:10.939292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:10.958806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:10.960131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:11.022329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:11.023855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:11.087401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:11.105564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 47/95] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:11.202532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 49/96] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:11.208601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:11.351604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:11.352208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aa31b-38f1-415a-9df0-7c76e7d97575 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:11.381964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:11.402612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:11.403286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.403562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.403837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.405033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:11.406505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:11.408415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 48/97] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:11.414731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:11.543656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:11.577486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3582-fd96-4b47-93b9-0696455243ce None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:11.578147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49acd91-f84f-40ec-9d9b-f0a6fdaddc58 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:11.617263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:11.617263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.617263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.617263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.617672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] RBAC: Adding query filter params (role.id=4fe0dc8c5db444d78f100f81099859c7, scope.project.id=9916f327817043fab7ba2144738659a1, user.id=ad176173d8984450b1076196825032c1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:11.617672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:11.619240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140b77-a53a-4af8-a588-80d9d0c4400f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:11.628006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 49/98] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/role_assignments?user.id=ad176173d8984450b1076196825032c1&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:11.628006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5536f8-fc53-4a19-a859-8e14c7616ecb None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.628006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5536f8-fc53-4a19-a859-8e14c7616ecb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.628423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5536f8-fc53-4a19-a859-8e14c7616ecb None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.629875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 50/99] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:11.630658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5536f8-fc53-4a19-a859-8e14c7616ecb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:11.667993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:11.668666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.668943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.669212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.697707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-0d8dce85-b021-4aa3-8fe0-3083b1afd748 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:11.710032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:11.710969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:11.711823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:11.714479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 50/100] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:11.716761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a524e-ce82-487b-a322-ae8a602e9684 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.717050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a524e-ce82-487b-a322-ae8a602e9684 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.717325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a524e-ce82-487b-a322-ae8a602e9684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.742889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b01a524e-ce82-487b-a322-ae8a602e9684 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:11.744918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b01a524e-ce82-487b-a322-ae8a602e9684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:11.786533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5536f8-fc53-4a19-a859-8e14c7616ecb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:11.818479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:11.823483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 51/101] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:11.825049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371bef-aea1-4473-a98f-7b48b95c6faf None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.825342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371bef-aea1-4473-a98f-7b48b95c6faf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.825620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6371bef-aea1-4473-a98f-7b48b95c6faf None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.831447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 52/102] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:11.831447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6371bef-aea1-4473-a98f-7b48b95c6faf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:11.834275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:11.839845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:11.840704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.841120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.841914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.857027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-feef3e6c-bdc3-4df6-8ae4-6dd04c6d1f69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:11.863192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:11.864057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:11.864834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 51/103] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:11.865959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30d95-b98f-4aac-b42e-6291f1189372 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.865959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30d95-b98f-4aac-b42e-6291f1189372 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.866102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e30d95-b98f-4aac-b42e-6291f1189372 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.868203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 52/104] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:11.870044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:11.870044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:11.870044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:11.890421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:11.893841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:11.966063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6371bef-aea1-4473-a98f-7b48b95c6faf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:11.983189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 53/105] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:11.983189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.009683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.010695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.011315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.011603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.012295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:12.012606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.015104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33caca-73ac-4376-913e-b461ce1fee7e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.024632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 53/106] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.026129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.026424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.026710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.049939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:12.052005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:12.092035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e85591-828e-4265-b275-ac37649ce8bd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.110019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.122478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.130212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.131038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.131438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.131713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.139608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 54/107] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:12.139729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:12.139729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.139887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.139955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.147357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 54/108] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.148017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.260260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b4d9c-af7e-4794-ad4a-fb5c15e23f49 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.260547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c23e-70d6-4db5-963b-470304cf94c7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.294467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.295100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.295411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.295684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.296808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.298350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.299737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.300106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 55/109] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.300785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.300884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.301470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.301889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.303142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.304913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.306666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 55/110] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.307641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.356628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.366184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.405570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed1f77-90f5-4b5e-9bb7-186659ae2078 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.414152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925992fd-d4a2-43be-995e-23c1e48d2dd6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.438903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.439567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.439847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.440123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] PATH_INFO: `/v3/users/f73fee2d56184c7e89f5b43c52d8ccce` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.446053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.446681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.446962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.447267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.453028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] RBAC: Authorizing `identity:get_user(user_id=f73fee2d56184c7e89f5b43c52d8ccce)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.454937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.456591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 56/111] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/users/f73fee2d56184c7e89f5b43c52d8ccce => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.458299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:12.458512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:12.463834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:12.464718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:12.465493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 56/112] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:12.466118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.559446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f5ee-c296-4d7d-9018-84ccb1b966ec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.561643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18d9-a8eb-4e12-b14b-423cb53b968e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.568913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.580098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.594584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.595224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.595510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.595787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.597964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.598687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.599005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.599326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.599981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.601439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.631764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 57/113] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:11:12 2022] POST /identity/v3/projects => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:12.632618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.668835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.670338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.762910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf7bb-2703-454b-90d0-71d185c5171f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.774573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 57/114] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:12.775502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.801773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.813819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.814589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.814937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.815327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.827680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:12.833109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:12.833987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:12.834747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 58/115] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:12.835425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.859488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:12.889762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba41915-b4c8-4361-8a0c-b7ba4f146c36 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.926522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.927231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.927475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.927753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.928886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.930442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:12.936539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 58/116] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:12.936539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:12.950302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28ca8c-3ef9-4e8c-a5b0-ab7430eb3272 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:12.990341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:12.990960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:12.991316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:12.991548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:12.992219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:12.992527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:12.993924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.002895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 59/117] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.008328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.025888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.040734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4add8-1ff1-4b82-ae73-44024dff72eb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.075279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.075911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.076194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.076458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.077582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.079093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.081358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 59/118] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:12 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.084065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.097155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.118111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed93d8-7f17-470e-a624-eb91bb4c16a4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.154931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.155519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.155801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.156045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.156788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:13.157060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.158481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.172361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 60/119] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.172361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.193290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2921831-5d0a-4901-a02f-7c4c010434d1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.231078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.232120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.232606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.232881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/users/f73fee2d56184c7e89f5b43c52d8ccce/roles/0dc731dcaf134a5698df55c793613c16` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.234856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, user_id=f73fee2d56184c7e89f5b43c52d8ccce, role_id=0dc731dcaf134a5698df55c793613c16)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.236372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.259202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 60/120] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:13 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/users/f73fee2d56184c7e89f5b43c52d8ccce/roles/0dc731dcaf134a5698df55c793613c16 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:13.259202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.276681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.295422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d601f7-4838-4ec6-be63-6f24fad1c17d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.341941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.342695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.343037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.343393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] PATH_INFO: `/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.344910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] RBAC: Authorizing `identity:get_user(user_id=ec20b39fa45c4429b8c7ce4fb9b19e90)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.346496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.348627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 61/121] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/users/ec20b39fa45c4429b8c7ce4fb9b19e90 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.349289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.363512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.384976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d1a1b-bfa8-4be2-ac34-7d642ca26520 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.428164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.428809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.429165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.429507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.430349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] RBAC: Adding query filter params (role.id=45f94767c7a04887a34b82d23eb15d91, scope.project.id=9916f327817043fab7ba2144738659a1, user.id=ad176173d8984450b1076196825032c1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:13.430725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.432442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.446867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 61/122] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/role_assignments?user.id=ad176173d8984450b1076196825032c1&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=45f94767c7a04887a34b82d23eb15d91 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.446867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.501445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b73f94-b8c5-4ab0-999c-ebab36106763 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.543508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.544183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.544468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.544751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] PATH_INFO: `/v3/users/ad176173d8984450b1076196825032c1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.546170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] RBAC: Authorizing `identity:get_user(user_id=ad176173d8984450b1076196825032c1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.547893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.549654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 62/123] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/users/ad176173d8984450b1076196825032c1 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:13.556520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.566068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.605657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aab9f2-e561-4f81-b998-b7b1ded9e38f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.682752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.683304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.683588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.683856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.703828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.728121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d3592-8be0-439f-bab6-d94c3fed4522 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:13.763627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.763845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:13.764446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:13.764772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:13.764992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:13.766353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.842861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:13.844448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:13.854204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.878913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 62/124] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:13.886007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:13.937683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:13.966270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 63/125] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:13.976186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:14.005042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b794ac7-bcfd-4834-a118-58400262a2a6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:14.049094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:14.050248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.050670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.051079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.052533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:14.054712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:14.057022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 63/126] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:14.064244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:14.096180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0f6f7-5712-4cfa-bbc1-5a4c9e613ddc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:14.115848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:14.136266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:14.137336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.137815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.138256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] PATH_INFO: `/v3/projects/03689a2def334ebf8fda976106f097ad` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.139599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] RBAC: Authorizing `identity:get_project(project_id=03689a2def334ebf8fda976106f097ad)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:14.141291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:14.143480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 64/127] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:13 2022] GET /identity/v3/projects/03689a2def334ebf8fda976106f097ad => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:14.152068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:14.173851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03c194-af9f-4c00-ad8a-aa11dae4df02 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:14.206972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:14.218271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:14.219412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.220217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.220912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.221977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=9916f327817043fab7ba2144738659a1, user.id=ec20b39fa45c4429b8c7ce4fb9b19e90) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:14.222458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:14.224655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bada3-e7d4-4eb1-b80f-c88859163a45 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:14.232470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 64/128] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:14 2022] GET /identity/v3/role_assignments?user.id=ec20b39fa45c4429b8c7ce4fb9b19e90&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:14.272896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e446bc-27c9-4b56-84ff-91306f4ce81d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:14.311398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:14.312406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.313043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.313570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.314583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] RBAC: Adding query filter params (role.id=4fe0dc8c5db444d78f100f81099859c7, scope.project.id=03689a2def334ebf8fda976106f097ad, user.id=ad176173d8984450b1076196825032c1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:14.315107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:14.317110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882afdc-bacc-4583-82fb-c9cbade89174 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:14.324999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 65/129] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:14 2022] GET /identity/v3/role_assignments?user.id=ad176173d8984450b1076196825032c1&scope.project.id=03689a2def334ebf8fda976106f097ad&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:14.872069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0ee-edf5-4aeb-86bb-195500af9e6a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.872565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0ee-edf5-4aeb-86bb-195500af9e6a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.872636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0ee-edf5-4aeb-86bb-195500af9e6a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.874008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 65/130] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:14.879611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:14.880014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:14.880097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:14.899955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:14.901142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:14.970537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 66/131] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:15.234238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e0ee-edf5-4aeb-86bb-195500af9e6a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:15.337426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e0ee-edf5-4aeb-86bb-195500af9e6a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:15.350272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:15.371583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:15.372206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:15.372482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:15.372749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:15.373874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:15.375416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2db137-0d4d-4def-8a79-16b2877b0a4e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:15.377203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 66/132] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:15.383013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:15.425554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:15.485093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97767698-b104-416b-9cbe-d11db3019c71 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:15.521012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:15.521606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:15.521884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:15.522144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:15.522806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:15.524316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609c00-21da-4a91-aeb3-9b7cedc4ef10 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:15.553505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 67/133] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:15 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:15.582175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091b54-1773-4733-95f9-4cb9038cd919 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:15.582465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091b54-1773-4733-95f9-4cb9038cd919 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:15.582746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57091b54-1773-4733-95f9-4cb9038cd919 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:15.584197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 67/134] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:15.590271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:15.590559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:15.590832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:15.612196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:15.613444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:15.691015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 68/135] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:15.954094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57091b54-1773-4733-95f9-4cb9038cd919 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:16.071008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57091b54-1773-4733-95f9-4cb9038cd919 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:16.106662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:16.107330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:16.107613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:16.107883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:16.119304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:16.124873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:16.125765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:16.126667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 68/136] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:16.132017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:16.160408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:16.237867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6733-fd61-45cd-a174-734473897f60 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:16.272534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:16.273167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:16.273443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:16.273713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:16.274391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:16.274688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:16.276112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:16.285215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 69/137] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:16.290897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:16.335139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:16.402037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a270e1-c9b2-462a-b400-1d3854230168 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:16.436651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:16.437295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:16.437608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:16.437910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] PATH_INFO: `/v3/users/f73fee2d56184c7e89f5b43c52d8ccce` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:16.439058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] RBAC: Authorizing `identity:get_user(user_id=f73fee2d56184c7e89f5b43c52d8ccce)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:16.440563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:16.442200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 69/138] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:16 2022] GET /identity/v3/users/f73fee2d56184c7e89f5b43c52d8ccce => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:16.448062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:16.542842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:16.556863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eaf6b-3269-4742-be48-16f392a481b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:16.592527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:16.593150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:16.593429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:16.593700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:16.665058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:16.666661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:16.757150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:16.776904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 70/139] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:16.778851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:16.879904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12111503-5cbc-45b1-baf8-b637ae64d054 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:16.889419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:16.916231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:16.917279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:16.917751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:16.918219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:16.919561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:16.921209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2917b9-144c-4dc3-ac63-ef41c9d78dd9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:16.923499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 70/140] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:16 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:16.931277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:17.033335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26976880-ebd5-4bf5-aecb-3c6fcdecac77 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:17.069773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:17.070437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:17.070720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:17.070988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:17.071825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] RBAC: Adding query filter params (role.id=0dc731dcaf134a5698df55c793613c16, scope.project.id=9916f327817043fab7ba2144738659a1, user.id=f73fee2d56184c7e89f5b43c52d8ccce) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:17.072127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:17.073584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41442d-2520-440f-8ba9-4bc442b138c7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:17.087341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 71/141] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:16 2022] GET /identity/v3/role_assignments?user.id=f73fee2d56184c7e89f5b43c52d8ccce&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=0dc731dcaf134a5698df55c793613c16 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:17.793310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee3e7-79ea-466d-b5ad-5899cb772704 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:17.793847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee3e7-79ea-466d-b5ad-5899cb772704 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:17.793847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee3e7-79ea-466d-b5ad-5899cb772704 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:17.795255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 71/142] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:17.802031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b145-df52-405d-b066-026707f14aad None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:17.802509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b145-df52-405d-b066-026707f14aad None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:17.802632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b145-df52-405d-b066-026707f14aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:17.824404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc71b145-df52-405d-b066-026707f14aad None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:17.825677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc71b145-df52-405d-b066-026707f14aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:17.877316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:17.897666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 72/143] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:17.932773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b56d46-7904-49ef-a3ec-70e21940f2ed None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:17.933027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b56d46-7904-49ef-a3ec-70e21940f2ed None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:17.933539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b56d46-7904-49ef-a3ec-70e21940f2ed None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:17.934876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 72/144] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:17.940242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18ac29-460f-45f9-917d-5068153c6d28 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:17.940546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18ac29-460f-45f9-917d-5068153c6d28 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:17.940826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18ac29-460f-45f9-917d-5068153c6d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:17.961197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0b18ac29-460f-45f9-917d-5068153c6d28 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:17.962489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0b18ac29-460f-45f9-917d-5068153c6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:18.035720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 73/145] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:18.077773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9879be-2428-4a9a-ae1e-c4408a6ce2e2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.078022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9879be-2428-4a9a-ae1e-c4408a6ce2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.078594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9879be-2428-4a9a-ae1e-c4408a6ce2e2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.080332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 73/146] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:18.085919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.086210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.086481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.105154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:18.106333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:18.157481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9879be-2428-4a9a-ae1e-c4408a6ce2e2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.177927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 74/147] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:18.257590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9879be-2428-4a9a-ae1e-c4408a6ce2e2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.290369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.290998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.291306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.291580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.295841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.302347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:18.307262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:18.308186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:18.309020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 74/148] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:18.313958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.344021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.350834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.405773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0fdb59-6065-4011-88c3-6497c6a4c27d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.415969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bcd2e-1863-4677-85fe-112e4c833e73 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.442050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.442696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.442973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.443272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.450667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.451300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.451576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.451842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.452509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:18.452806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.453172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:18.454218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:18.458548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:18.459519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 75/149] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:18.460505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.465177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 75/150] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:18.466025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.557823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.559080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948845d-5153-4de2-ae10-ba6158bd123d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.560676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.561979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc824-966a-4cc1-ba6a-6e59ff67d6ec None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.593197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.593868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.594144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.594412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.595529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.595774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.596391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.596670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.596944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.597028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.597613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:18.597918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.598708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 76/151] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:18.599317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.599408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.607458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 76/152] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:18.608226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.703437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57edc-0765-45cd-8fc7-1c5b62d0cc1c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.713044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea854570-19dd-4ed7-8c57-6358cd2783e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.736705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.737339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.737617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.737888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] PATH_INFO: `/v3/users/e414b57053ba4b65a5c6d701c1332796` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.746776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.747423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.747707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.747974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.748630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.750147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.752610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] RBAC: Authorizing `identity:get_user(user_id=e414b57053ba4b65a5c6d701c1332796)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.753996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.755663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 77/153] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/users/e414b57053ba4b65a5c6d701c1332796 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:18.760483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.770997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 77/154] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:18 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:18.776391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.790794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.803612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:18.876586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f184b0-b933-4125-ac10-23011e866e20 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.895047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c484f-10b4-443e-97e6-322668b1e679 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:18.918493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.919189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.919470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.919728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] PATH_INFO: `/v3/users/e414b57053ba4b65a5c6d701c1332796` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.920942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] RBAC: Authorizing `identity:get_user(user_id=e414b57053ba4b65a5c6d701c1332796)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.922512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.924202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 78/155] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/users/e414b57053ba4b65a5c6d701c1332796 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:18.929956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:18.940902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:18.941549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:18.941820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:18.942467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:18.944010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:18.945757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:18.953067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 78/156] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:18.955603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:19.038645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:19.045932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d617b9-d6de-4e00-b773-6a2f4b5c6892 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:19.055956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:19.079817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b14f89-2b7b-4989-9dd7-c9bc6a233c98 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:19.091171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:19.091829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:19.092143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:19.092379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:19.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:19.095033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:19.098031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 79/157] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:18 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:19.104729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:19.128767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:19.130265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:19.130265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:19.130265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/e414b57053ba4b65a5c6d701c1332796/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:19.133200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, user_id=e414b57053ba4b65a5c6d701c1332796, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:19.134691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09680ddc-2ec5-4534-90c5-7469935847e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:19.150643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 79/158] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:18 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/e414b57053ba4b65a5c6d701c1332796/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:19.231425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b002f-841b-44c7-a320-4d2a264d6e93 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:19.270617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:19.271719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:19.272210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:19.272732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/e414b57053ba4b65a5c6d701c1332796/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:19.275372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, user_id=e414b57053ba4b65a5c6d701c1332796, role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:19.277118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1695f-e44e-4cd3-a8fe-0aa772f5d994 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:19.294137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 80/159] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:19 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/e414b57053ba4b65a5c6d701c1332796/roles/45f94767c7a04887a34b82d23eb15d91 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:21.043565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a743-6085-4e08-b0b9-4e33f8df01ea None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.044119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a743-6085-4e08-b0b9-4e33f8df01ea None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.044119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a743-6085-4e08-b0b9-4e33f8df01ea None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.045486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 80/160] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:21.050912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8844ff-e8cc-4637-a9f3-5b5ff148d177 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.051348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8844ff-e8cc-4637-a9f3-5b5ff148d177 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.051522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8844ff-e8cc-4637-a9f3-5b5ff148d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.070941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6d8844ff-e8cc-4637-a9f3-5b5ff148d177 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:21.072183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6d8844ff-e8cc-4637-a9f3-5b5ff148d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:21.090967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:21.114594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a337-e933-4cde-8934-239226edba15 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.114876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a337-e933-4cde-8934-239226edba15 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.115171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a337-e933-4cde-8934-239226edba15 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.116544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 81/161] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:21.121755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddba0-68a6-4741-8f25-b9a8bed64704 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.122040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddba0-68a6-4741-8f25-b9a8bed64704 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.122312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dddba0-68a6-4741-8f25-b9a8bed64704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.144401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f4dddba0-68a6-4741-8f25-b9a8bed64704 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:21.145610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4dddba0-68a6-4741-8f25-b9a8bed64704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:21.148690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 81/162] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:21.213150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 82/163] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:21.355124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda2de-8a63-4ded-a054-bb23ba11e390 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.355509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda2de-8a63-4ded-a054-bb23ba11e390 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.355630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eda2de-8a63-4ded-a054-bb23ba11e390 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.357437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 82/164] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:21.363841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065423-2b98-4878-8aa8-e9fd6355db5a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.364478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065423-2b98-4878-8aa8-e9fd6355db5a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.364938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065423-2b98-4878-8aa8-e9fd6355db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.386464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d065423-2b98-4878-8aa8-e9fd6355db5a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:21.388164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d065423-2b98-4878-8aa8-e9fd6355db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:21.422609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eda2de-8a63-4ded-a054-bb23ba11e390 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.475418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 83/165] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:21.476691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693650-8d99-45fc-9ebd-5c0dd82201b6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.476973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693650-8d99-45fc-9ebd-5c0dd82201b6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.477233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9693650-8d99-45fc-9ebd-5c0dd82201b6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.478591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 84/166] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:21.483822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.484388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.484966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.492735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:21.505984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:21.507460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:21.533831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eda2de-8a63-4ded-a054-bb23ba11e390 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:21.569346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:21.569977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.570253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.570520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.578237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 85/167] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:21.581152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:21.584106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:21.585972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:21.586883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:21.587743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 83/168] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:21.594586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.614602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:21.696162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395bd87-60e0-4580-9cb3-4f65b9caf8c9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:21.727187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcba0fa-539b-4e44-ad09-7ed940f7a193 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:21.731643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:21.732372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.732718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.733047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.734224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:21.735827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:21.737496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 86/169] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:21.742758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.745607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:21.766115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:21.766722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.766995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.767297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.767965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:21.768260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:21.769664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:21.778230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 84/170] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:21.778827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.856297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f3f32-4e99-4cf8-8195-a57500c5a2dd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:21.861544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:21.884651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd395e-1741-4a85-ba2f-742d40cdb1e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:21.889426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:21.890076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.890354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.890703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.900441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:21.905239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:21.906092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:21.906883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:21.911243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 87/171] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:21.911938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.918230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:21.918841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:21.919114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:21.919417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:21.920067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:21.921522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:21.942940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 85/172] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:21 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:21.943758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:21.994110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.038857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982c9f3-da56-49d1-8060-9b4c1f7d277e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.064209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83543cae-711f-4b7c-ab15-c51e2e321611 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.075775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.076430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.076704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.076970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] PATH_INFO: `/v3/users/50eab505254b48ba887930a55c7f24bc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.095141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] RBAC: Authorizing `identity:get_user(user_id=50eab505254b48ba887930a55c7f24bc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.096884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.098694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 88/173] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/users/50eab505254b48ba887930a55c7f24bc => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:22.103843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.108448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.115237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504238a-487b-4075-917c-003f55830bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.115951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504238a-487b-4075-917c-003f55830bcf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.116341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504238a-487b-4075-917c-003f55830bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.116912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504238a-487b-4075-917c-003f55830bcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.128976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504238a-487b-4075-917c-003f55830bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9504238a-487b-4075-917c-003f55830bcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:22.134327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:22.135321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:22.136110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 86/174] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:22.140089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504238a-487b-4075-917c-003f55830bcf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.222324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13f062-4c3d-491b-9658-cddbd1f8509f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.258124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.265316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504238a-487b-4075-917c-003f55830bcf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.278166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.279428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.279428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.279428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.281743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:22.281743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.281927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.292414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 89/175] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:22.293203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.309333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.310077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.310431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.310773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.312039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.313667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.315600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 87/176] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:22.321215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.389799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.430271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda4ba-3fc0-4712-9ab5-e3af2e14a601 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.439742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0f677-d186-4138-8b05-085a7c64cd80 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.477252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.478605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.479181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.479554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.479836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.479934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] PATH_INFO: `/v3/users/e414b57053ba4b65a5c6d701c1332796` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.480382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.480899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.481309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] RBAC: Authorizing `identity:get_user(user_id=e414b57053ba4b65a5c6d701c1332796)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.481990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:22.482555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.483084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.484363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.484939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 88/177] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/users/e414b57053ba4b65a5c6d701c1332796 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:22.490149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.501328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 90/178] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:22.502516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.556997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.632867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66787efb-2e36-4547-a0e5-59d44bb0c860 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.634550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e27c7-9ebb-4026-b14c-91e565757771 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.665645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.675795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.676413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.676518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.676682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.676951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.677861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.678328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.678763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/50eab505254b48ba887930a55c7f24bc/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.681172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, user_id=50eab505254b48ba887930a55c7f24bc, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.682846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.699336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 89/179] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:22 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/50eab505254b48ba887930a55c7f24bc/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:22.706965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.771724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.774146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.878371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:22.943587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56fc7-9657-4f61-9c36-d7a8727ea036 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:22.958454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 91/180] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:22.964738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:22.988226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:22.989222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:22.989674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:22.990140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] PATH_INFO: `/v3/users/e414b57053ba4b65a5c6d701c1332796` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:22.991465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] RBAC: Authorizing `identity:get_user(user_id=e414b57053ba4b65a5c6d701c1332796)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:22.993164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:22.994977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 90/181] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/users/e414b57053ba4b65a5c6d701c1332796 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:23.000657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:23.072881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:23.099409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc2ebe-990f-4860-b53d-8565356a253c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:23.124006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88a21e-956a-4393-90ff-ad4ee359b1d5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:23.147524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:23.148464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:23.148740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:23.149057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:23.150506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:23.152775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:23.155358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 92/182] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:23.161075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:23.167202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:23.168078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:23.168551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:23.169035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:23.176295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:23.244854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:23.246812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:23.266406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29b64f-f99b-4433-b9d5-e4b0d26958b7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:23.301852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:23.302832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:23.302832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:23.303081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:23.303906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b, user.id=e414b57053ba4b65a5c6d701c1332796) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:23.304213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:23.305738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:23.307253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:23.313553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 93/183] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:23 2022] GET /identity/v3/role_assignments?user.id=e414b57053ba4b65a5c6d701c1332796&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:23.360476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 91/184] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 25 13:11:23.366054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:23.488280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:23.524440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4500-bde6-4f40-ba53-53719543433a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:23.569626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:23.570264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:23.570539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:23.570810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:23.572081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:23.573532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27edfd26-fcd4-409c-bcfe-f5a306c958b8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:23.575380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 94/185] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:23 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:23.581716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:23.687004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490fb10-563e-4656-86ea-9b61e3d807e6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:23.723723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:23.724339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:23.724611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:23.724873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:23.725613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] RBAC: Adding query filter params (role.id=45f94767c7a04887a34b82d23eb15d91, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b, user.id=e414b57053ba4b65a5c6d701c1332796) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:23.725915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:23.727405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f4017-9d26-4bdb-b3a0-420a430cc774 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:23.735604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 92/186] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:23 2022] GET /identity/v3/role_assignments?user.id=e414b57053ba4b65a5c6d701c1332796&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=45f94767c7a04887a34b82d23eb15d91 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:24.509697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5be29-a7c0-47b5-b6ef-2c386ff54d46 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:24.510247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5be29-a7c0-47b5-b6ef-2c386ff54d46 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:24.510247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5be29-a7c0-47b5-b6ef-2c386ff54d46 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:24.511693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 95/187] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:24.517044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:24.517486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:24.517562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:24.536053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:24.537261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:24.552651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:24.568212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44e81-1451-4ae3-8801-eb990d64b158 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:24.568502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44e81-1451-4ae3-8801-eb990d64b158 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:24.568768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44e81-1451-4ae3-8801-eb990d64b158 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:24.570179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 96/188] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:24.575514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:24.576634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:24.576910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:24.597243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:24.598445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:24.607105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 93/189] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:24.666324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 97/190] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:24.873889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:24.930209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:24.981211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc118f8b-1d2d-40fa-96cd-cb699794f002 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.017516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.018842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.019592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.020043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.020291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.021561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.023052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.024755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 94/191] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:25.030152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.045665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ed446-1b23-458a-b2a1-5c070ca4f8ef None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.050651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.086249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.086891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.087197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.087468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.098380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5360d956-9d87-4a1a-87a1-13ece64683dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:25.103349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:25.104260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:25.105076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 98/192] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:25.105748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb489-3634-448c-8df3-c8a5d278ab30 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.106424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb489-3634-448c-8df3-c8a5d278ab30 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.106424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb489-3634-448c-8df3-c8a5d278ab30 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.107558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 99/193] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:25.111631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb489-3634-448c-8df3-c8a5d278ab30 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.135076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0724e1b-001d-4e2e-ba69-2b1eda185575 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.171116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.171820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.172074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.172344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.173005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.174469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998c85d-26cb-4122-ac3b-1b3a14ef817b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.196100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 95/194] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:25 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:11:25.196100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.196100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.196100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.211096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb489-3634-448c-8df3-c8a5d278ab30 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.227202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:25.228779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:25.248234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.248887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.249134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.249399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.250171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:25.250473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.253578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.264407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 100/195] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:25.270003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.317136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 96/196] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:25.388051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84159f-6ad7-459d-aa11-c63328fa468a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.423287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.423957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.424238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.424511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] PATH_INFO: `/v3/users/3d3a838f2b9d461e918cf579fbf70451` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.438887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] RBAC: Authorizing `identity:get_user(user_id=3d3a838f2b9d461e918cf579fbf70451)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.440418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.442081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 101/197] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/users/3d3a838f2b9d461e918cf579fbf70451 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:25.447767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.452284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.556023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cab96-77d2-4aa6-90a0-9be53fb7c8f0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.589894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.594204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.594867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.595141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.595440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.596588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.598064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.599958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 97/198] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:25.605345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.652570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.698395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660fa26-db0c-4ef7-a4fd-f5c3ba87642f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.698599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.735483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.736302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.736503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be525-358a-4c62-9fc8-4f381b9e0162 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.736769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.737134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.748182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:25.753556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:25.754489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:25.755312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 102/199] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:25.761803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:25.773981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.774967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.775490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.775948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/3d3a838f2b9d461e918cf579fbf70451/roles/0dc731dcaf134a5698df55c793613c16` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.778181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, user_id=3d3a838f2b9d461e918cf579fbf70451, role_id=0dc731dcaf134a5698df55c793613c16)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.779911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.795769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 98/200] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:25 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/users/3d3a838f2b9d461e918cf579fbf70451/roles/0dc731dcaf134a5698df55c793613c16 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:11:25.904844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c9e9b-9d8d-4a50-a135-91111c3eb2d3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:25.921139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:25.943469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:25.944095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:25.944368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:25.944633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:25.945312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:25.945607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:25.947025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:25.957602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 103/201] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:25.963533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:26.073911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c882d-1ea5-40e4-8aaf-72758fe0c139 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:26.105876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:26.113445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:26.114108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:26.114469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:26.114745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] PATH_INFO: `/v3/users/50eab505254b48ba887930a55c7f24bc` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:26.116114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] RBAC: Authorizing `identity:get_user(user_id=50eab505254b48ba887930a55c7f24bc)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:26.119069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:26.120956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 99/202] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:25 2022] GET /identity/v3/users/50eab505254b48ba887930a55c7f24bc => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:26.126523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:26.237168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1afa96-a742-483f-93ba-e375c9dadb97 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:26.272258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:26.273238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:26.273733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:26.274176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:26.325960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:26.355056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:26.356869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:26.456194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 104/203] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:26.462111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:26.567044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed7e9a-163d-4c17-bb8e-10c10b43fdcf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:26.604643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a14000-db92-4066-af12-6031987fa683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:26.605622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14000-db92-4066-af12-6031987fa683 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:26.606070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14000-db92-4066-af12-6031987fa683 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:26.606507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14000-db92-4066-af12-6031987fa683 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:26.607848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a14000-db92-4066-af12-6031987fa683 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:26.609559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a14000-db92-4066-af12-6031987fa683 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:26.611448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 100/204] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:26 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:26.617604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:26.698942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:26.723661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51f03-350a-48c6-bec6-fbd7a11fe486 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:26.757736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:26.758344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:26.758620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:26.758889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:26.759658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] RBAC: Adding query filter params (role.id=4fe0dc8c5db444d78f100f81099859c7, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b, user.id=50eab505254b48ba887930a55c7f24bc) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:26.759970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:26.761509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4414435-c14d-4c12-9cb9-9808d6efd0d1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:26.767453 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 105/205] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:26 2022] GET /identity/v3/role_assignments?user.id=50eab505254b48ba887930a55c7f24bc&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:27.561682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c536b3-dce6-4eb8-8df1-7b2c1e17a366 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:27.561929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c536b3-dce6-4eb8-8df1-7b2c1e17a366 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:27.562375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c536b3-dce6-4eb8-8df1-7b2c1e17a366 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:27.563968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 101/206] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:27.569137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:27.569422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:27.569653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:27.581973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae419b2-ab77-472c-b2ea-1e4291f9d9e4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:27.582506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae419b2-ab77-472c-b2ea-1e4291f9d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:27.582506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae419b2-ab77-472c-b2ea-1e4291f9d9e4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:27.584017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 102/207] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:27.588904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:27.589182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:27.589410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:27.589504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:27.590654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:27.607913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:27.609084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:27.671209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 106/208] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:27.673906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:27.683189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 103/209] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:27.941235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:27.946837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.019514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.052942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980390-fef4-4395-87a6-88d21feafb88 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.053408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9f45b-f4aa-40ed-a1a2-7629f8073eca None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.088399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.088601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.089034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.089254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.089344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.089529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.089618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.089797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.090879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.092369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab3236-9dc0-460f-8cf8-f3b2a912e7ca None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.094016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 104/210] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:28.095075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25526-eac6-408f-9c89-7e2d6f6007a1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.095378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25526-eac6-408f-9c89-7e2d6f6007a1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.095649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25526-eac6-408f-9c89-7e2d6f6007a1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.096836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 105/211] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:28.099442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a25526-eac6-408f-9c89-7e2d6f6007a1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.103838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-38246b7d-9511-48e0-b077-7e039aabe8f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:28.108709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:28.109666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:28.110558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 107/212] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:28.111254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.111995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.111995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.129617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:28.130793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:28.165766 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.195799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 108/213] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:28.196539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.202590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a25526-eac6-408f-9c89-7e2d6f6007a1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.244213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.244812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.245085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.245347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.246009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.247473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.255772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.271017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 106/214] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:28 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:11:28.303875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d61426-1ec7-4c4d-ab89-ab3d890b09c5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.342894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.343708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.344037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.344309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.344997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:28.345297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.346778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.357310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 109/215] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:28.363235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.460278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.471245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf3fe4-76ae-4d01-81cd-b3ae0a2a8113 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.496366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.512207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.512870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.513150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.513421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] PATH_INFO: `/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.528135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfbe00fa2f748a69dd8fcdb0f3a26ef)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.529631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.532458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 107/216] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:28.538847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.583982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422b548-8d8f-4d5e-aa2d-5f5d75548fa7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.608433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.628134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.629033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.629843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.631495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.647141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:28.652576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:28.653465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:28.654271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 110/217] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:28.659576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.661859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33fbe-0c9b-4c73-8341-d75ed0efad53 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.702655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.703394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.703668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.703935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.705046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] RBAC: Adding query filter params (role.id=4fe0dc8c5db444d78f100f81099859c7, user.id=bbfbe00fa2f748a69dd8fcdb0f3a26ef, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:28.705352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.706871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.712662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 108/218] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/role_assignments?user.id=bbfbe00fa2f748a69dd8fcdb0f3a26ef&scope.system=all&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:28.782815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b01e-bc2a-40e4-b8fb-8a407403c434 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.822397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.823022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.823326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.823593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.824278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:28.824574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.826239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.836884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 111/219] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:28.843016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:28.902267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:28.957092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc80f-f016-4ec0-9790-ee77030a6419 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:28.991524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:28.992081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:28.992356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:28.992622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] PATH_INFO: `/v3/users/3d3a838f2b9d461e918cf579fbf70451` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:28.993750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] RBAC: Authorizing `identity:get_user(user_id=3d3a838f2b9d461e918cf579fbf70451)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:28.995222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:28.997173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 109/220] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:28 2022] GET /identity/v3/users/3d3a838f2b9d461e918cf579fbf70451 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:29.002590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:29.049855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:29.106590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c2e1f-a0cf-430b-b77e-65dcdd0f0bcf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:29.140961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:29.141579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:29.141851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:29.142117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:29.215541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:29.217015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:29.262304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:29.321413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 112/221] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:29.327447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:29.435053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471455-cd9e-48fd-a546-2464cda879e9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:29.448582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:29.470220 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:29.470908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:29.471277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:29.471604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:29.472770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:29.474228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72161210-e742-4694-99d8-e8a8f20bb6b9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:29.475913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 110/222] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:29 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:29.481968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:29.582821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fece-d9dc-429e-b7a4-f2866582ec42 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:29.620525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:29.621189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:29.621469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:29.621736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:29.622480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] RBAC: Adding query filter params (role.id=0dc731dcaf134a5698df55c793613c16, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b, user.id=3d3a838f2b9d461e918cf579fbf70451) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:29.622787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:29.625868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7cf493-6d39-4ab3-ac66-57323905e41f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:29.635286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 113/223] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:11:29 2022] GET /identity/v3/role_assignments?user.id=3d3a838f2b9d461e918cf579fbf70451&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=0dc731dcaf134a5698df55c793613c16 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:30.629734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524963-152d-4389-8122-a296591e2829 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.630144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524963-152d-4389-8122-a296591e2829 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.630360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524963-152d-4389-8122-a296591e2829 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.631905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 111/224] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:30.637242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abd44-965b-436e-a0a8-cfa5a2017aa6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.637679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abd44-965b-436e-a0a8-cfa5a2017aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.637755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abd44-965b-436e-a0a8-cfa5a2017aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.639578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89baf5-e620-49bf-89b5-e880ed2c2137 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.639843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89baf5-e620-49bf-89b5-e880ed2c2137 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.640144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89baf5-e620-49bf-89b5-e880ed2c2137 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.641597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 112/225] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:30.646307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d442e8-3514-4671-ab2d-a4923b43a443 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.646601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d442e8-3514-4671-ab2d-a4923b43a443 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.646854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d442e8-3514-4671-ab2d-a4923b43a443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.657906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bd0abd44-965b-436e-a0a8-cfa5a2017aa6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:30.659794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bd0abd44-965b-436e-a0a8-cfa5a2017aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:30.662037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:30.665040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-81d442e8-3514-4671-ab2d-a4923b43a443 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:30.666202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-81d442e8-3514-4671-ab2d-a4923b43a443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:30.732702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 113/226] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:30.735792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 114/227] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:30.871097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378bdcf-5532-424e-ae24-fee0f7b2ce7c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.871579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378bdcf-5532-424e-ae24-fee0f7b2ce7c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.871907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378bdcf-5532-424e-ae24-fee0f7b2ce7c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.873346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 114/228] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:30.878787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:30.878997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:30.879433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:30.898030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:30.899340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:30.971322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 115/229] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:30.996909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378bdcf-5532-424e-ae24-fee0f7b2ce7c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.005181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.074770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.105878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378bdcf-5532-424e-ae24-fee0f7b2ce7c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.111590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635af338-94aa-44ed-9f7a-bea806b0f2d1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.117200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.142475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.143041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.143333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.143599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.144673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.146117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.146712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.147379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.147868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.147978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 115/230] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:31.148059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.153084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.158874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:31.163830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:31.164694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:31.165463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 116/231] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:31.170754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.258267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088ae7-d18c-4dbe-aaad-d8cdf6167a68 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.271992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8794fc-eb2f-4b02-a9b3-33ef2eb714c2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.292020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.292610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.292884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.293152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] PATH_INFO: `/v3/groups` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.293777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.295248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.297872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.306338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.306982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.307295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.307560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.308220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:31.308520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.309890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.311422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 116/232] 10.222.0.36 () {60 vars in 1134 bytes} [Tue Oct 25 13:11:31 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:11:31.312295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.318553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 117/233] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:31.328563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.365080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.438622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd741c6-9eb5-46ed-8e4d-46f5bdde648e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.438823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d1c3-2f9d-4620-b13a-a73f584125af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.476334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.476987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.477269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.477553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] PATH_INFO: `/v3/users/2d3b4a129ecc4a239a142569626a75a0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.478699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.479380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.479665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.479931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.490848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:31.494171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] RBAC: Authorizing `identity:get_user(user_id=2d3b4a129ecc4a239a142569626a75a0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.495690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:31.496216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:31.497055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:31.497859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 117/234] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:31.498544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 118/235] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/users/2d3b4a129ecc4a239a142569626a75a0 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:31.503238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.503385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.556782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.607407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.608894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a2d8-ac1e-4179-b57d-5e4afd803c3a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.611316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e497-a7dd-41c2-b3f9-9b83f9672536 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.648848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.649495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.649717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.649997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.650149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.650726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] RBAC: Adding query filter params (role.id=0dc731dcaf134a5698df55c793613c16, user.id=2d3b4a129ecc4a239a142569626a75a0, scope.system=all) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:31.650822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.650997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.651100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.651286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.651946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:31.652242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.652513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.653613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.658134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 119/236] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/role_assignments?user.id=2d3b4a129ecc4a239a142569626a75a0&scope.system=all&role.id=0dc731dcaf134a5698df55c793613c16 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:31.662494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 118/237] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:31.668162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.779176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecc70e-1d2b-4b89-b18b-a67c4f549073 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.795353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:31.821689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.822408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.822675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.822942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] PATH_INFO: `/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.824237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfbe00fa2f748a69dd8fcdb0f3a26ef)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.825735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926b43-335d-453e-8a51-ce25d0f5f318 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.827613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 120/238] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:31 2022] GET /identity/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:31.834199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:31.939427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4cbd3-2ed8-43df-a36e-cd063f040290 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:31.974962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:31.975691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:31.975967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:31.976234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] PATH_INFO: `/v3/system/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:31.977623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bbfbe00fa2f748a69dd8fcdb0f3a26ef, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:31.979027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0132ea-74ef-479e-a2a4-9e8a2dc41609 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:31.995185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 119/239] 10.222.0.36 () {58 vars in 1334 bytes} [Tue Oct 25 13:11:31 2022] PUT /identity/v3/system/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:33.483685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73c83-55a4-4b04-94f6-fa7f1ab800a3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:33.484293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73c83-55a4-4b04-94f6-fa7f1ab800a3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:33.484293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73c83-55a4-4b04-94f6-fa7f1ab800a3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:33.485580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 121/240] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:33.491224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecb5-276a-4b12-ae0d-ceffe086e1b8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:33.491662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecb5-276a-4b12-ae0d-ceffe086e1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:33.491803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2ecb5-276a-4b12-ae0d-ceffe086e1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:33.511189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-81a2ecb5-276a-4b12-ae0d-ceffe086e1b8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:33.512410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-81a2ecb5-276a-4b12-ae0d-ceffe086e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:33.532160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:33.587982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 120/241] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:33.773723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808853a6-2b90-4d22-b8af-6825f0447cb5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:33.773958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808853a6-2b90-4d22-b8af-6825f0447cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:33.774255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808853a6-2b90-4d22-b8af-6825f0447cb5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:33.775799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 122/242] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:33.781156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:33.781417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:33.781707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:33.800616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:33.801783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:33.852189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808853a6-2b90-4d22-b8af-6825f0447cb5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:33.877251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 121/243] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:33.957821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808853a6-2b90-4d22-b8af-6825f0447cb5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:33.996002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:33.996665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:33.996941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:33.997213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:33.998436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.000019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9af18-18ef-45a8-8c16-e37f8439293e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.001923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 123/244] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:34.007550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.097150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.101410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf434-e842-4f35-80d4-c479eb127c16 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.102037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf434-e842-4f35-80d4-c479eb127c16 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.102967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf434-e842-4f35-80d4-c479eb127c16 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.104860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 124/245] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:34.109807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.110324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.110604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.113752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdcf79-ad57-4c36-bb3f-ca741da32fbd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.129047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:34.130244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:34.148043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.148656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.148932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.149199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] PATH_INFO: `/v3/groups` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.149840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.151519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.167457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 122/246] 10.222.0.36 () {60 vars in 1134 bytes} [Tue Oct 25 13:11:34 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:34.168448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.179112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.207399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 125/247] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:34.290802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cba8f2-f813-48ab-a83e-52d0661fe550 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.327141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.327848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.328122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.328397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.341308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:34.346820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:34.347877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:34.348691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 123/248] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:34.353996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.458386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca2fa2-a9a5-44ec-a437-2e0062964b20 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.477412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.482656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.504925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.505814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.505814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.506077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.506759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:34.507058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.508472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.519000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 126/249] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:34.525535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.580162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.616707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfc92-8d65-4520-8d05-5e198f35fd85 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.647772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f9b2f-81e4-401f-bd22-c5acaf7bd8af None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.667850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.668498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.668780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.669043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.683291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:34.689014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:34.689915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:34.690693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 124/250] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:34.696677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.696677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.696677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.696677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] PATH_INFO: `/v3/users/2d3b4a129ecc4a239a142569626a75a0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.697021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] RBAC: Authorizing `identity:get_user(user_id=2d3b4a129ecc4a239a142569626a75a0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.697021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.698539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.700557 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 127/251] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/users/2d3b4a129ecc4a239a142569626a75a0 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:34.706935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:34.794317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.835705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf0e22-599e-4d51-a42e-0295b12e1c49 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.842952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a73e3-fe6a-465c-8ddc-7e4a067f2c90 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:34.846715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:34.877359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.878086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.878481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.878852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] PATH_INFO: `/v3/system/users/2d3b4a129ecc4a239a142569626a75a0/roles/0dc731dcaf134a5698df55c793613c16` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.880684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d3b4a129ecc4a239a142569626a75a0, role_id=0dc731dcaf134a5698df55c793613c16)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.884519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.890905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:34.892036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:34.892577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:34.893097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:34.893983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:34.894448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:34.895864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 128/252] 10.222.0.36 () {58 vars in 1334 bytes} [Tue Oct 25 13:11:34 2022] PUT /identity/v3/system/users/2d3b4a129ecc4a239a142569626a75a0/roles/0dc731dcaf134a5698df55c793613c16 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:34.896887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:34.910654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 125/253] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:34.914221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:35.053776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc80a8f-42b7-4091-8019-c83af8f0be88 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:35.099680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:35.100357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:35.100606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:35.100875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] PATH_INFO: `/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:35.102092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] RBAC: Authorizing `identity:get_user(user_id=bbfbe00fa2f748a69dd8fcdb0f3a26ef)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:35.103669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f01f5-ae41-42d9-8185-8d41baef7937 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:35.107615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 129/254] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:34 2022] GET /identity/v3/users/bbfbe00fa2f748a69dd8fcdb0f3a26ef => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:35.111988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:35.234652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb396d1-a466-4bf0-b45f-e014d75bf24b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:35.263879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:35.276960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:35.277647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:35.277930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:35.278200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:35.278954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] RBAC: Adding query filter params (role.id=4fe0dc8c5db444d78f100f81099859c7, user.id=bbfbe00fa2f748a69dd8fcdb0f3a26ef, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:35.279285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:35.280762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fbe11-1d9d-48ea-8a54-f541fa4b4950 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:35.298056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 126/255] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Oct 25 13:11:35 2022] GET /identity/v3/role_assignments?user.id=bbfbe00fa2f748a69dd8fcdb0f3a26ef&scope.system=all&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:36.656933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f2740-026a-4f0b-9f4e-1a2d41859afc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:36.657545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f2740-026a-4f0b-9f4e-1a2d41859afc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:36.657545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f2740-026a-4f0b-9f4e-1a2d41859afc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:36.659101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 130/256] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:36.668577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e468de-a4e7-4df3-a3c4-2298903c9211 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:36.669456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e468de-a4e7-4df3-a3c4-2298903c9211 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:36.669977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e468de-a4e7-4df3-a3c4-2298903c9211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:36.700245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d7e468de-a4e7-4df3-a3c4-2298903c9211 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:36.701827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7e468de-a4e7-4df3-a3c4-2298903c9211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:36.777540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 127/257] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:11:37.238980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cbc5d-97b5-41e5-a545-d294f72754ed None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.239686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cbc5d-97b5-41e5-a545-d294f72754ed None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.240195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cbc5d-97b5-41e5-a545-d294f72754ed None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.242037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 131/258] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:37.247252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.247804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.248270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.276507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:37.278098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:37.280628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cbc5d-97b5-41e5-a545-d294f72754ed None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.295137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 128/259] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:11:37.304756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cbc5d-97b5-41e5-a545-d294f72754ed None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.314971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.315655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.315931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.316195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.336081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:37.341136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:37.342028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:37.342847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 132/260] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:37.349804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.365102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c632736-dd51-4994-923e-f36280f4f8f0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.371373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.372326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.372794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.373262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.374134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:37.374725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.380878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.380878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:11:37.380878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:11:37.382131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.392197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 129/261] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.397525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.413198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae9a2b-4d56-44c1-abb1-1518100edb70 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.423168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.423973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.424407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.424806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.435630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.437716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.437716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:11:37.437716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:11:37.439273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.442060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 133/262] 10.222.0.36 () {56 vars in 1201 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.446622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.457600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47709e-92dd-4c65-9163-57dd17bebc68 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.470502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.471186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.471459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.471735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.473925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.476085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f8d46-59e3-4c20-bf30-585aa3a40fe1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.477938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 130/263] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.483510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.494089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641d9b-4459-4fa2-95fe-a675fb3f49c3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.503128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.504073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.504553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.505007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/groups/fb903921034e465da22d0cad44d4e2f8/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.507064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, group_id=fb903921034e465da22d0cad44d4e2f8, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.508311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.508311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:11:37.508311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:11:37.510605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49c0e1-95b4-4dd0-8640-85d796f11854 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.535789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 134/264] 10.222.0.36 () {58 vars in 1463 bytes} [Tue Oct 25 13:11:37 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/groups/fb903921034e465da22d0cad44d4e2f8/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:11:37.538974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da772-ac40-4ef0-9085-d99e19199c22 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.539304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da772-ac40-4ef0-9085-d99e19199c22 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.539582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da772-ac40-4ef0-9085-d99e19199c22 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.540968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 131/265] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:37.550448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.550753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.550995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.580578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:37.581804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:37.586811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:37.653818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 135/266] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:11:37.713384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da772-ac40-4ef0-9085-d99e19199c22 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.737127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da772-ac40-4ef0-9085-d99e19199c22 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.741607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.742253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.742529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.742797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.760909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:37.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:37.767345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:37.768528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 132/267] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:37.773729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.789226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56df43-f728-40e9-88c7-ecac10af3db5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.797686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.798724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.799215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.799705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.800588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:37.801084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.802315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.802315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:11:37.802315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:11:37.803650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.819606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 136/268] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.825300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.836047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b139d9-6b48-4abb-a632-e92670d0ef57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.847957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.848628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.848901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.849171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.856119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.857439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.857439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:11:37.857439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:11:37.858680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.860819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 133/269] 10.222.0.36 () {56 vars in 1201 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.866428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.880247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3eb87-6742-49ae-9200-5fcf8d887588 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.891778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.892815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.893279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.893736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.895134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.897452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.899691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 137/270] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:11:37 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:37.905073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:37.919421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91232d9-d5eb-4d6a-b718-a035d5c95884 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:37.927232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:37.927987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:37.928261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:37.928537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1/groups/fb903921034e465da22d0cad44d4e2f8/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:37.936967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9916f327817043fab7ba2144738659a1, group_id=fb903921034e465da22d0cad44d4e2f8, role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:37.938177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:11:37.938177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:11:37.938177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:11:37.940210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:37.965687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 134/271] 10.222.0.36 () {58 vars in 1463 bytes} [Tue Oct 25 13:11:37 2022] PUT /identity/v3/projects/9916f327817043fab7ba2144738659a1/groups/fb903921034e465da22d0cad44d4e2f8/roles/45f94767c7a04887a34b82d23eb15d91 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:38.181412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.192468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bab24b-57b3-4a73-88b0-5d705f2d2b42 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:38.271081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:38.272126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.272593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.273069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.286085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:38.296701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:38.297658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:38.298430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:38.300800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 138/272] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 120 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:38.312301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.328491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8e2d-754a-41ca-96d8-72db55446949 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:38.339894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:38.340531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.340808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.341103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.341819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:38.342123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:38.343835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:38.353625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 135/273] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:11:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:38.359699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.370295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf56297-05e6-4bc3-990e-1af4a312fc72 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:38.379118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:38.380048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.380513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.380993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.382403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:38.384594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08e9851-4a0f-4d98-8058-ea6ccdc63893 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:38.386395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 139/274] 10.222.0.36 () {56 vars in 1201 bytes} [Tue Oct 25 13:11:38 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:38.392490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.403052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86e181-4f0e-4999-a397-20d318aec8ec admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:38.403378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545dc48-004d-4902-990d-f65c6f3bbee5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.403471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545dc48-004d-4902-990d-f65c6f3bbee5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.403724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545dc48-004d-4902-990d-f65c6f3bbee5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.405311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 140/275] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:38.410348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.410625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.410891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.420454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:38.421112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.421423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.421727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.422931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:38.425044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:38.427018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 136/276] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:11:38 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:38.431709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:38.432606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.433334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:38.448123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53aeb6-b656-4598-8679-856c495584dc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:38.458055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:38.458735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:38.459026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:38.459313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/groups/fb903921034e465da22d0cad44d4e2f8/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:38.461232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, group_id=fb903921034e465da22d0cad44d4e2f8, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:38.463949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:38.498212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 137/277] 10.222.0.36 () {58 vars in 1463 bytes} [Tue Oct 25 13:11:38 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/groups/fb903921034e465da22d0cad44d4e2f8/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:38.510709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 141/278] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:38.872147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:38.934870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:38.991728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418a173-552a-4620-93bc-06a8fee11bd0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.031636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.032284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.032567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.032836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.047379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a2df5ab5-463a-46fc-a8a3-911da38807c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:39.052812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:39.053690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:39.054478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 138/279] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:39.061634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:39.074563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:39.147561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e8cc-4f0c-4f9f-9417-c9da9374e925 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.147750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e8cc-4f0c-4f9f-9417-c9da9374e925 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.148012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee15e8cc-4f0c-4f9f-9417-c9da9374e925 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.149674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 139/280] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:39.159049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.159384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.159657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.192157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:39.193385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:39.196497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8df7d-6166-4e64-9545-3e864b5bd29d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.241335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.241960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.242279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.242934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.243659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:39.243961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:39.245407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:39.257758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 142/281] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:39.264134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:39.336209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 140/282] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:11:39.352701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:39.394170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3185d-6cbc-4aac-bb9c-723dba2e2e9e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.435552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.436112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.436376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.436642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] PATH_INFO: `/v3/users/2d3b4a129ecc4a239a142569626a75a0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.437882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] RBAC: Authorizing `identity:get_user(user_id=2d3b4a129ecc4a239a142569626a75a0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:39.439440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c1c37-f19a-497d-b541-7b94cdfbc026 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:39.441334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 143/283] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:11:39 2022] GET /identity/v3/users/2d3b4a129ecc4a239a142569626a75a0 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:39.446993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:39.514312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:39.558565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4285c-f2b9-45bf-bd91-b07cac6f2795 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.600993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.601605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.601881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.602149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.602886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] RBAC: Adding query filter params (role.id=0dc731dcaf134a5698df55c793613c16, user.id=2d3b4a129ecc4a239a142569626a75a0, scope.system=all) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:39.603209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:39.604677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a831fe8-8c39-43ce-8994-38a99654c402 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:39.620991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 141/284] 10.222.0.36 () {56 vars in 1308 bytes} [Tue Oct 25 13:11:39 2022] GET /identity/v3/role_assignments?user.id=2d3b4a129ecc4a239a142569626a75a0&scope.system=all&role.id=0dc731dcaf134a5698df55c793613c16 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:39.779771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f63aa-8414-4cca-b659-8b93cc83d7a8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.780186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f63aa-8414-4cca-b659-8b93cc83d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.780365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f63aa-8414-4cca-b659-8b93cc83d7a8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.781868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 144/285] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:39.788291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.788576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.788849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.810371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:39.811660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:39.819604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 142/286] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 25 13:11:39.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f63aa-8414-4cca-b659-8b93cc83d7a8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:39.933170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f63aa-8414-4cca-b659-8b93cc83d7a8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.940527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.941186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.941456 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.941720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.956308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:39.961374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:39.963462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:39.965347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 145/287] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:11:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:39.973014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:39.984793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddaa97c-f032-4315-aa36-380ff21876ea None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:39.989315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:39.989991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:39.990264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:39.990532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:39.991283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:39.991585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:39.993210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.005304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 143/288] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Oct 25 13:11:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.011716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.023209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d892a-37da-4af5-904b-27d1850f33da admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.027947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.028691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.028970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.029241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.031472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.033402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.035490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 146/289] 10.222.0.36 () {56 vars in 1201 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.041026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.051690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5283ed0-af05-47c8-89d7-ed548685f1aa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.058243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.058937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.059291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.059565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.060763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.062919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.065086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 144/290] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.070758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.082012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c77d0-b791-4c5e-b6be-9dce26f397e9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.086683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.087367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.087699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.087972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/groups/fb903921034e465da22d0cad44d4e2f8/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.093000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14e9aadb7a44d16a99f76fc33cff57b, group_id=fb903921034e465da22d0cad44d4e2f8, role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.095891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.115865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 147/291] 10.222.0.36 () {58 vars in 1463 bytes} [Tue Oct 25 13:11:40 2022] PUT /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b/groups/fb903921034e465da22d0cad44d4e2f8/roles/45f94767c7a04887a34b82d23eb15d91 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:40.179931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.191956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22fab8-66ac-4920-a6cf-5526267315a3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.260516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.261176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.261446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.261711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.273528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:40.279388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:40.281823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:40.283880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 145/292] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 102 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:40.287211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.301484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9220-96ae-4409-b9d6-33c3be2c61d0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.306744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.307400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.307716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.307980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.308677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:40.309012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.310699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.320904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 148/293] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.326668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.338369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad31d8-b32b-49aa-ac3c-93bcda02ee61 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.345090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.345741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.346014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.346325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] PATH_INFO: `/v3/groups/aba7f618cf0b48f4830cf2c9e4b901d2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.354098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] RBAC: Authorizing `identity:get_group(group_id=aba7f618cf0b48f4830cf2c9e4b901d2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.356131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.357880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 146/294] 10.222.0.36 () {56 vars in 1201 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/groups/aba7f618cf0b48f4830cf2c9e4b901d2 => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.363554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.374239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055c3e-647c-45df-b78e-b9cae845d304 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.379087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.379849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.380119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.380383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] PATH_INFO: `/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.381579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] RBAC: Authorizing `identity:get_project(project_id=e7cf0d7d46da4af3b19a1ac43ed2227b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.383615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e02fa-18d5-4819-b6ea-79b7988587f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.385685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 149/295] 10.222.0.36 () {56 vars in 1207 bytes} [Tue Oct 25 13:11:40 2022] GET /identity/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b => generated 344 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:40.391677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:40.402194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa20485-c2a5-43fe-891f-3bbb2643cd90 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:40.409697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:40.410385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:40.410705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:40.410976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] PATH_INFO: `/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b/groups/aba7f618cf0b48f4830cf2c9e4b901d2/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:40.413169 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7cf0d7d46da4af3b19a1ac43ed2227b, group_id=aba7f618cf0b48f4830cf2c9e4b901d2, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:40.416268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b9b3b-ea8a-420b-a7eb-0817d5d4cc94 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:40.440998 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 147/296] 10.222.0.36 () {58 vars in 1463 bytes} [Tue Oct 25 13:11:40 2022] PUT /identity/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b/groups/aba7f618cf0b48f4830cf2c9e4b901d2/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:41.178323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16972c-ddfb-416e-ae3f-9722066442f1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.179093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16972c-ddfb-416e-ae3f-9722066442f1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.179093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16972c-ddfb-416e-ae3f-9722066442f1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.180504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 150/297] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:41.186209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabda1d-f161-434d-ac05-fac3de82961e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.186652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabda1d-f161-434d-ac05-fac3de82961e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.186833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabda1d-f161-434d-ac05-fac3de82961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.216214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bfabda1d-f161-434d-ac05-fac3de82961e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:41.217417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bfabda1d-f161-434d-ac05-fac3de82961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:41.232830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:41.300171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 148/298] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:41.439428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7b447-099a-4383-bedd-808699a25521 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.440068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7b447-099a-4383-bedd-808699a25521 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.440593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7b447-099a-4383-bedd-808699a25521 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.442427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 151/299] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:41.447625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.447912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.448190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.469063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5fff-9dda-4866-8323-ea8a1e257dd2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.469369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5fff-9dda-4866-8323-ea8a1e257dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.469600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5fff-9dda-4866-8323-ea8a1e257dd2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.471220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 152/300] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:11:41.476131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.476406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.476673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.486822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:41.492871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:41.503653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:41.504838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:41.623927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 149/301] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:41.624862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:41.645109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 153/302] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:41.711358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:41.729429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1216eec-0551-4359-a8db-65048d25a5ff None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:41.767715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:41.768359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.768629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.768895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.781638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:41.786655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:41.787565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:41.788382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 150/303] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:41.793865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:41.844990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:41.934093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:41.943176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad4ce0-6ac3-4cb8-813f-4c65e907ba09 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:41.982904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:41.983242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:41.983496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:41.983802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:41.984497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] RBAC: Adding query filter params (name=member) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:41.984775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:41.986144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.003251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 154/304] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:42.012161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.079247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58859d-d106-4b03-9851-abd3d57300e9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.144656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.145305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.145569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.145831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.161293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.163971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:42.169950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:42.170873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:42.171770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 151/305] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:42.178211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.179878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.210956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27e891-d32d-484b-b593-d4c28a93ebae None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.260258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.261089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.261503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.261884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.298669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5180146-fcad-42d0-a9d2-da2ebfde58be None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.341517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.342381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.342657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.343224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.344344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:42.344654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:42.346543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.350774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:42.353595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.356751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 152/306] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:42.360068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.444441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.462470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.480352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 155/307] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 25 13:11:42.481278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.496275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787733b5-6af2-481a-a689-99dc7dc75c55 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.549118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.549687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.549959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.550235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.561971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:42.570242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:42.571166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:42.571979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 153/308] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:42.581258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.615436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3ef4e-ec88-480a-88ed-2d43c776f8cb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.657719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.658782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.659277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.659774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.712099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0c560-434f-4325-b1e4-966f25cb9e32 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.732677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.740716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:42.750957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:42.753220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.754932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.755659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.755940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.756208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.758382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:42.759879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.761542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 154/309] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:42.762866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.883311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 156/310] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:42.884571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.893326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475759-1fd3-4488-894d-e6a27f8faee4 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:42.941009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:42.941637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:42.942044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:42.942444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:42.943130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:42.944654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:42.946052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:42.955972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 155/311] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:42.956715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:42.986639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.013735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126a31d-46b8-4c9f-8ba3-cc4359a086f9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.031361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.058893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.059973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.060487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.060955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.062474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.064247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.066110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 157/312] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.067052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.100769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a18e-fc8d-4ccb-ab97-9de702cf89e6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.156000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.157162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.157631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.158111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.159521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.161176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.163057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 156/313] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:42 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.164062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.214485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81285f98-2788-4225-b703-a29b626fe906 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.264403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.265175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.265433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.265696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.279258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1607b3-6f2f-4f58-8b35-62c5449787cc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.297240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.302719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.320304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.320920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.321204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.321476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.322224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] RBAC: Adding query filter params (group.id=fb903921034e465da22d0cad44d4e2f8, role.id=4fe0dc8c5db444d78f100f81099859c7, scope.project.id=9916f327817043fab7ba2144738659a1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:43.322531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.324022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72be4e-d85a-4987-8191-1e616f36dd42 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.331544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 157/314] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/role_assignments?group.id=fb903921034e465da22d0cad44d4e2f8&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.332775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acad9e-bcd4-4ed9-97d4-7b32f346092b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.333051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acad9e-bcd4-4ed9-97d4-7b32f346092b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.333322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acad9e-bcd4-4ed9-97d4-7b32f346092b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.334562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 158/315] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:43.339485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acad9e-bcd4-4ed9-97d4-7b32f346092b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.344655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.346147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85c9b6-8184-4b0c-ba06-d277bf4e7f65 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.486239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acad9e-bcd4-4ed9-97d4-7b32f346092b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.499237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 158/316] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:43.500490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f88c-8913-4b4a-a957-eab3ef29328c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.500764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f88c-8913-4b4a-a957-eab3ef29328c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.501039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f88c-8913-4b4a-a957-eab3ef29328c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.502507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 159/317] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:43.503468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.503751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.504469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.526413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:43.527842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:43.544189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.545624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.546047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.546423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.548053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.549504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8eb79-6689-4465-9129-acff74976bd1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.551129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 159/318] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.552351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.552626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.552895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.572090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:43.573306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:43.589605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.612426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 160/319] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:43.613600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.638745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.665340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 160/320] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:43.666342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.748410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cddb1-504e-4e1f-a20f-a831137e3297 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.792588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.793234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.793513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.793787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] PATH_INFO: `/v3/projects/9916f327817043fab7ba2144738659a1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.794918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] RBAC: Authorizing `identity:get_project(project_id=9916f327817043fab7ba2144738659a1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.796529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.798208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 161/321] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/projects/9916f327817043fab7ba2144738659a1 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.803860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.814476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90555c-7099-4726-9942-89b651b352a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.863135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.863804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.864077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.864519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.865587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] RBAC: Adding query filter params (group.id=fb903921034e465da22d0cad44d4e2f8, role.id=4fe0dc8c5db444d78f100f81099859c7, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:43.865904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.867680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.876691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.878786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 161/322] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/role_assignments?group.id=fb903921034e465da22d0cad44d4e2f8&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=4fe0dc8c5db444d78f100f81099859c7 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:43.902361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:43.943077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16671a-66c7-4c04-9c34-f9bc6a81a4bb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:43.971429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:43.984668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:43.985324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:43.985595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:43.985864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:43.987060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:43.988571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:43.990550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 162/323] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:43.995233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.051255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c842fb-b53c-42da-8ba0-12a6bcc5e2ce None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.105055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.105261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.105517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.105718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.119340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:44.126913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:44.128009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:44.128796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:44.130654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0a481-ad1b-4193-8b63-dbb337bcc6e4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.131261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 162/324] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:44.133151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.156937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.177458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.178096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.178377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.178709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.189507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:44.195100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:44.196367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 163/325] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:44.197384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.266499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8030855-42d9-424b-9700-3c022d41b953 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.273720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.313245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.313842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.314046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.314235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437c19-e2db-4b8f-abc7-a26ac1349a47 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.314626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.315042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] RBAC: Adding query filter params (group.id=fb903921034e465da22d0cad44d4e2f8, role.id=45f94767c7a04887a34b82d23eb15d91, scope.project.id=9916f327817043fab7ba2144738659a1) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:44.315423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.316889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.327313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 163/326] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/role_assignments?group.id=fb903921034e465da22d0cad44d4e2f8&scope.project.id=9916f327817043fab7ba2144738659a1&role.id=45f94767c7a04887a34b82d23eb15d91 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:44.327313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.363107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.364047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.364333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.364601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.365787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:44.366908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.370894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.380343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 164/327] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:44.387256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.445334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.456006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c764d-024b-480a-9d0b-65c9ab62aa9a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.506799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.507455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.507927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.508202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.509309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:44.509664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.511066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.521253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 164/328] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:44.527728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.533380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b283fe0-800e-423c-ae51-73f6d54634f1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.559676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.575298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.575838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.576117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.576396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.647767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803b10-c383-4aea-84f6-3d61b8596ef1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.659898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.661387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.690182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.690906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.691217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.691494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.716909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.771790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.773471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.789362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 165/329] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 25 13:11:44.795093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.808999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.891307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 165/330] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:11:44.896005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.916526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774879f-de54-43c3-be38-9a1b29361734 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:44.948133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:44.948751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:44.949025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:44.949294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] PATH_INFO: `/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:44.950386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] RBAC: Authorizing `identity:get_project(project_id=f14e9aadb7a44d16a99f76fc33cff57b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:44.951864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:44.953558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 166/331] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/projects/f14e9aadb7a44d16a99f76fc33cff57b => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:44.959450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:44.970594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:44.994859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528057f-65bc-4632-b520-44640024d3fc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:45.023880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:45.029122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:45.029754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:45.030030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:45.030297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] PATH_INFO: `/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:45.031527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] RBAC: Authorizing `identity:get_project(project_id=e7cf0d7d46da4af3b19a1ac43ed2227b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:45.032975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:45.034608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 166/332] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/projects/e7cf0d7d46da4af3b19a1ac43ed2227b => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:45.040117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:45.057049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0cd3c-b55c-4d39-aa12-842dc0f0fb14 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:45.091925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:45.092555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:45.092828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:45.093138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] PATH_INFO: `/v3/groups/fb903921034e465da22d0cad44d4e2f8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:45.094224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] RBAC: Authorizing `identity:get_group(group_id=fb903921034e465da22d0cad44d4e2f8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:45.095691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:45.097269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 167/333] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Oct 25 13:11:44 2022] GET /identity/v3/groups/fb903921034e465da22d0cad44d4e2f8 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:45.103116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:45.137448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2754e-5e7d-4c69-8ee6-79fc2c0e098d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:45.170046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:45.170607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:45.170881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:45.171187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] PATH_INFO: `/v3/groups/aba7f618cf0b48f4830cf2c9e4b901d2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:45.172333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] RBAC: Authorizing `identity:get_group(group_id=aba7f618cf0b48f4830cf2c9e4b901d2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:45.173757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:45.175386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 167/334] 10.222.0.36 () {56 vars in 1180 bytes} [Tue Oct 25 13:11:45 2022] GET /identity/v3/groups/aba7f618cf0b48f4830cf2c9e4b901d2 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:45.177809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:45.181310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:45.200205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc21d0-38a7-4abf-be73-03aa775faae1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:45.233182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:45.233862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:45.234137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:45.234402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:45.235126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] RBAC: Adding query filter params (group.id=fb903921034e465da22d0cad44d4e2f8, role.id=45f94767c7a04887a34b82d23eb15d91, scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:45.235463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:45.236901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122342f4-22ef-4cf1-b4e2-cc23d90fdc44 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:45.242633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 168/335] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Oct 25 13:11:45 2022] GET /identity/v3/role_assignments?group.id=fb903921034e465da22d0cad44d4e2f8&scope.project.id=f14e9aadb7a44d16a99f76fc33cff57b&role.id=45f94767c7a04887a34b82d23eb15d91 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:45.246971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:45.286444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363200e-2ac3-42d5-9f08-7466401177f7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:45.324421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:45.325130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:45.325272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:45.325574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:45.326330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] RBAC: Adding query filter params (group.id=aba7f618cf0b48f4830cf2c9e4b901d2, role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=e7cf0d7d46da4af3b19a1ac43ed2227b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:45.326637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:45.328334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84187de5-f04d-4ffb-a161-72f821caa88a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:45.335535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 168/336] 10.222.0.36 () {56 vars in 1376 bytes} [Tue Oct 25 13:11:45 2022] GET /identity/v3/role_assignments?group.id=aba7f618cf0b48f4830cf2c9e4b901d2&scope.project.id=e7cf0d7d46da4af3b19a1ac43ed2227b&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:48.069249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f18a6-9fe9-47a8-81fc-d7ed528e1494 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.070250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f18a6-9fe9-47a8-81fc-d7ed528e1494 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.070745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f18a6-9fe9-47a8-81fc-d7ed528e1494 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.072624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 169/337] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:48.078151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60242c-7e2d-468e-8df6-8d4b2679e8a5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.078986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60242c-7e2d-468e-8df6-8d4b2679e8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.079853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60242c-7e2d-468e-8df6-8d4b2679e8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.103370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6a60242c-7e2d-468e-8df6-8d4b2679e8a5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:48.104916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6a60242c-7e2d-468e-8df6-8d4b2679e8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:48.197726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 169/338] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:48.286384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942bd73-f74d-44b9-a8ec-88442ffcdc11 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.287032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942bd73-f74d-44b9-a8ec-88442ffcdc11 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.289264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942bd73-f74d-44b9-a8ec-88442ffcdc11 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.291425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 170/339] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:48.296931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.297514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.297992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.317941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:48.323925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:48.388482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03288-bcc9-4ebd-ba15-3728b13cd8b8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.388828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03288-bcc9-4ebd-ba15-3728b13cd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.389010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03288-bcc9-4ebd-ba15-3728b13cd8b8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.390722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 171/340] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:48.396207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.396493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.396765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.402230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:48.424429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 170/341] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:48.424845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:48.426334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:48.524721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 172/342] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:48.590885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:48.701159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:48.744261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41b310-07c5-4e01-9717-72bea00209a0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:48.782227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:48.783406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.783883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.784396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.792492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:48.796824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:48.798556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-91647844-6f07-4837-8d7a-1f175d7501b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:48.815522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:48.816484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:48.817341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:48.817341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:48.817341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:48.823692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 171/343] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:48.823814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dc1ae-920a-4ab7-83ba-c72ed93eb0f2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:48.825403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dc3d8-ec42-4555-8f22-060e32dfa79b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.825976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dc3d8-ec42-4555-8f22-060e32dfa79b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.826441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dc3d8-ec42-4555-8f22-060e32dfa79b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.828242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 172/344] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:48.834360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dc3d8-ec42-4555-8f22-060e32dfa79b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:48.860415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:48.860978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.861254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.861528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.873143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:48.874669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:48.887008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-7406ad0d-44c2-4d75-85f7-b93594c2181a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:48.889793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:48.891330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:48.892229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:48.892229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:48.892229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:48.892229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 173/345] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:48.892722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.893008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.893277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.914684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:48.915892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:48.945507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dc3d8-ec42-4555-8f22-060e32dfa79b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:48.981040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:48.981747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:48.982098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:48.982442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:48.987218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:48.987345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 174/346] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:48.987424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:48.987492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:48.987581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.004903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 173/347] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:49.008091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.046559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.094876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80c295-ad4e-4c82-9dfa-0330f039fcb4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.130967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.132128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.132541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.132939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.144685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96beb4-6ed7-426b-a0e5-5440edb0a6d6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.146629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.147472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.149357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:49.167479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:49.168580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:49.169437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:49.169437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:49.169437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:49.171664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 175/348] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:49.172692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.193185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.193817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.194098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.194635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.195373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:49.195681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.197789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.214586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 174/349] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:49.215883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.291300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015a002-b40f-4ab9-abe4-5c0b7a5b5364 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.298413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.318548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273188f-aedf-4fa9-86cd-0df50f6c5a6f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.328457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.329741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.330030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.330386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.330949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.332459 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.363181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 176/350] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:49 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:49.363181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.364734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.365488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.365841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.366240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.367198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:49.367662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.369163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.384849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 175/351] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:49.385874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.425186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.507030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cdaea-223c-4e3d-87e7-c7c7a9a5c136 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.520894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68158-d9a5-4c79-91ca-97678fdda839 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.550768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.551826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.552275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.552715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.553565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.555070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.572285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.573167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.574020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.574020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.582041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 177/352] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:49 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:49.582860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.587616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.589486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:49.609899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:49.610855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:49.611725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:49.611725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:49.611725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:49.613863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 176/353] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:49.616708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.623340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.723179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:49.725094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650c40f-bd71-454a-887f-c412c3168e6d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.749595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566316-4581-4dba-b7c1-2802d9447f78 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.765816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.766444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.766719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.766987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.767777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.769348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aa1a7-e33e-4ce4-93a9-06cea102b306 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.789027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.789833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.790239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.790561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.791469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:49.791808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.793411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:49.796657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 178/354] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:49 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:49.798116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bdf1-1b81-4c53-87bd-9b2c846e18d6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.798380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bdf1-1b81-4c53-87bd-9b2c846e18d6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.798649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bdf1-1b81-4c53-87bd-9b2c846e18d6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.800097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 179/355] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:49.806132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.806422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.806733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.807429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 177/356] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:49.815307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:49.832299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:49.833536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:49.914193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 180/357] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:49.949615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b82f-d0e1-4f82-af20-cd5c272a827b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:49.989937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:49.990492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:49.990776 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:49.991045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:49.991741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:49.993394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:50.004058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:50.024928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 178/358] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:49 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:50.202745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:50.207496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:50.317770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42983701-d84f-4cb8-9e80-ff0770e9ac9d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:50.355398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:50.356015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:50.356291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:50.356558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:50.357240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:50.358674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:50.368472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 25 13:11:50.368472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 25 13:11:50.368472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [parameters: {'id': 'bf2f909c529e4792a74f6ffeb2f7f047', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 25 13:11:50.368472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 25 13:11:50.385218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return ref.to_dict() Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:11:50.386138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 25 13:11:50.387507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 25 13:11:50.388353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:11:50.389289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:11:50.390386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'id': 'bf2f909c529e4792a74f6ffeb2f7f047', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:50.391878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 25 13:11:50.393577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:50.395106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 181/359] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Oct 25 13:11:50 2022] POST /identity/v3/roles => generated 144 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:50.395291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:50.524011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4eaf7-ef6d-4e63-804d-aa000ef5dd88 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:50.572172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:50.572739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:50.573015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:50.573285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:50.594149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ca8f9e22-ebd0-4bd3-bd8f-cd5a49a77955 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:50.599713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:50.601753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:50.603675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 179/360] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:11:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:50.616523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:50.712649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:50.755348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c662a-2978-4879-ae7b-6f9df1e38d34 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:50.800104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:50.800724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:50.800996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:50.801260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:50.801930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:50.802225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:50.803641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a676-7b66-4fde-a98b-ed814cbfe19f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:50.814715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 182/361] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:51.872566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f906-29b1-44ee-8b00-b246454eeebf None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:51.873264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f906-29b1-44ee-8b00-b246454eeebf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:51.873264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f906-29b1-44ee-8b00-b246454eeebf None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:51.874774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 180/362] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:51.879927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:51.880414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:51.880414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:51.904124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:51.905313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:51.996806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 183/363] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:52.424487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd74c4-b58a-4afa-a61d-67e195a1a498 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.424818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd74c4-b58a-4afa-a61d-67e195a1a498 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.425026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd74c4-b58a-4afa-a61d-67e195a1a498 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.426560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 181/364] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:52.427515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:52.431617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7834f-304f-4b11-a234-54c5816c943d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.431907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7834f-304f-4b11-a234-54c5816c943d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.432178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7834f-304f-4b11-a234-54c5816c943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.450786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46b7834f-304f-4b11-a234-54c5816c943d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:52.452039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46b7834f-304f-4b11-a234-54c5816c943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:52.493896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:52.514872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:52.539928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 182/365] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:52.540367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcf5b5-0fd1-443f-843e-8b848aade8e1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:52.541185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563c4d-1128-4ff2-aa91-b4a534ee2c93 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.541472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563c4d-1128-4ff2-aa91-b4a534ee2c93 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.543377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563c4d-1128-4ff2-aa91-b4a534ee2c93 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.544828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 183/366] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:52.550832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.551108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.551399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.589051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:52.589735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.590010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.590275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.590943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:52.592379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:52.600792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:52.606535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:52.607455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:52.612435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 184/367] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:52.617102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:52.732021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 184/368] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:52.790314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda596-5e84-4428-a3e3-bfb9debb92bf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:52.810375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:52.825777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:52.826450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:52.826716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:52.826979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:52.827779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:52.828074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:52.829897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:52.842455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 185/369] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:52.848421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:52.881356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:52.975746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51cf46-c6d0-4543-b771-bf6b60c98832 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:52.995417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.004172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d3e6-e53b-4c67-8fae-2d02c22124b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.025062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.026713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.029326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.029694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.043707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.043995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.044821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.045247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.045572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.045782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.056861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e9b1ac53-5a58-491d-b904-0df278655c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:53.062195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.063094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:53.064481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:53.064481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.064481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-51743061-14f8-4808-9eb4-e0eee362f1ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:53.064646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.065506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.066283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:53.066888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 185/370] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.066888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aae7f0-5a46-4e0c-a3a2-a262a23b4d77 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.066888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aae7f0-5a46-4e0c-a3a2-a262a23b4d77 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.066888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aae7f0-5a46-4e0c-a3a2-a262a23b4d77 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.066888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 186/371] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:53.067866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aae7f0-5a46-4e0c-a3a2-a262a23b4d77 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.071199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 186/372] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.073427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.073894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.074347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.093220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:53.094699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:53.131840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.166279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 187/373] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:53.167593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.179230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aae7f0-5a46-4e0c-a3a2-a262a23b4d77 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.222269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.223067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.223456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.223824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.238157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:53.245835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.246989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.247898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:53.248510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 187/374] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.249365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.286551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.295494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce9f6d-6e27-40d5-ba3a-105cdacbaf8d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.334365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.335447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.335965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.336393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.337260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:53.337748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.339908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.353307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 188/375] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:53.354440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.379366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbccc8-d89c-4ca9-b8c0-0f75ba880af6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.392082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.418406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.419045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.419344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.419861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.420540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:53.420858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.422992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.432847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 188/376] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:53.433641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.467736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057fe8-c98e-43ee-a5b7-e5ba3ddcdeed None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.502314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.502957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.503295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.503598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.504264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:53.504587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.505956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.515168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 189/377] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:53.515814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.531171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.547247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5f834-db5e-42aa-bfa8-07ff225cac5f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.583957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.584956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.585413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.585690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.594528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.596059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:11:53.610161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.611201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 189/378] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.612226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.630488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.637162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a1ff-96bb-4fd9-bb58-c24e8bfa73cf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.672194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.672929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.673275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.673613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.683922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:53.689705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.690639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 190/379] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.696593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.723203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b7591-c164-400d-859f-165dc3e908bd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.756705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.757266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.757540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.757807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.769210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.770702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.777170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:53.786292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.787403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:53.788398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:53.788398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.788398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:53.788398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 190/380] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.788398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.792752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f63b-4df4-441f-ac4d-4ff599c2b391 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.825466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.826101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.826382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.826650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.837375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.838839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:53.852933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:53.854005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:53.854887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:53.854887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:53.854887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:53.854887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 191/381] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:53.855101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.864729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:53.889760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8ec96-3af0-4899-abf3-2fa933470b06 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.921926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.922528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.922797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.923062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.923823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:53.924121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:53.925533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d7c09-d41a-431c-bd9f-0287976ce5b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:53.955584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ad5a-0d54-4d13-88f9-772cea3aaa65 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:53.965218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 191/382] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:53.966584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77ff5f-2017-41a8-9284-3d3ee9dfc9e8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.966871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77ff5f-2017-41a8-9284-3d3ee9dfc9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.967140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77ff5f-2017-41a8-9284-3d3ee9dfc9e8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.968620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 192/383] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:53.969183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77ff5f-2017-41a8-9284-3d3ee9dfc9e8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:53.996748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:53.997358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:53.997632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:53.997903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:53.998574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:53.998876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.000554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.008762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 192/384] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.009490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.026063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.071199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77ff5f-2017-41a8-9284-3d3ee9dfc9e8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.075430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.104306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.104908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.105185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.105450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.106098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.106389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.107806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.109611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427995e-b130-4874-ba32-2fd8fda74cde None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.118644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 193/385] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.119447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.147884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.148496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.148781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.149044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.149693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.149990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.151421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc041eb-5ee7-4739-9511-706576a1c994 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.162635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 193/386] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.163795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.164071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.164340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.181264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:54.182426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:54.220750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184253e-186b-48d7-9d00-2a3b2939868e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.237249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.248442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 194/387] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:54.262614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.262725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.262725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.262725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.262725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.269682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.271191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:54.286519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:54.287481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:54.288327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:54.288327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.288327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:54.288327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 194/388] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:54.288563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.317262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.365449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ecbbb-9ea1-4ad6-b8bd-df1bfa128f07 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.396442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e98e5-e7a8-4bd7-93ca-13e063cd0217 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.399545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.400177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.400449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.400710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.411183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.412836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:54.428043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:54.429029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:54.429874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:54.429874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.429874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:54.429874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 195/389] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:54.429874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.434384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.435020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.435325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.435596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.444984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.446847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:11:54.458371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:54.459493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:54.460401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 195/390] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:54.460960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.494119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.535130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab105a-0451-49ee-8e87-301d32bb0cbe None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.545117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.566173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d257-4393-4ded-9965-8b1cf2449019 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.567983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.568613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.568885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.569148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.576412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.577881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:11:54.590598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:54.591704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 196/391] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:54.597685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.601583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.602184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.602457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.602724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.603414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.603689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.605080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.617164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 196/392] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.618447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.698202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8939f-43b8-4130-97cb-f37673eadb9b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.702782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.716006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78c5ad-ebf6-4978-8daa-a10416c0404f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.733076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.733713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.733992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.734262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.734918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.735260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.736674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.748119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 197/393] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.748902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.753909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.754533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.754808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.755074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.755844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.756138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.757561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.774650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 197/394] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.775308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.782646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:54.853402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589068-6a09-4ea3-82cc-a8908167a9f1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.871662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb58767-900d-4516-9a61-556c69d91442 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:54.887698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.888322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.888595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.888861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.900039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.901515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.906361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:54.906970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:54.907383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:54.907655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:54.908384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:54.908691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:54.910122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:54.916729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:54.917808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:54.918646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:54.918646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:54.918646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:54.918646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 198/395] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:54.918646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.933372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 198/396] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:54.933372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:54.933372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.003672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.021893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5699c-33d2-4b36-ac95-bbd57a7ed001 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.029261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe9a3-6bbe-45b0-943d-e72b0a52142b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.055746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.056359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.056637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.056907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.062674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.063329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.063425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.063606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.063871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.064859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.070729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.072199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:11:55.076488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:55.077517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 199/397] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:55.078405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:11:55.088692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:55.089803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 199/398] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:55.092474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.152300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.178934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673a804-7ec8-4d32-b1d9-e8ac693b67cf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.191284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495c7-836b-4b00-a250-bbb08e2d82a1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.211811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.212437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.212715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.212985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.223531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.224058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.224663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.224939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.225064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.225170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.225843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:55.226141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.226681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.227506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.239162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 200/399] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:55.239991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:55.240594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:55.241477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:55.242319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:55.242319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:55.242319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:55.242319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 200/400] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:55.242540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.343319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bad49-5310-4506-b3db-00a47d4d59b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.351947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22981-b5dc-4a1c-839b-21540bb92de8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.376057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.376934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.377547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.377823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.378097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.378821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:55.379134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.380629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.387676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.388310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.388589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.388864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.397959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 201/401] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:55.398655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.403782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.405573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:55.420380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:55.421490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:55.422385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:55.422385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:55.422385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:55.422385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 201/402] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:55.422609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.458258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.501837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527605c-5a35-40b7-b9e4-91d3055c847e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.522231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d65-8825-4365-93d5-cb56f87fe0f2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.533925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.534527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.534804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.535073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.535835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:55.536137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.537591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.556878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.557486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.557765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.558036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.558683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.560159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.573247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 202/403] 10.222.0.36 () {56 vars in 1139 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:55.574088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.583540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 202/404] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:11:55 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:11:55.584332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.626317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.702989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd51c-078a-4778-b1a8-779b3f93e9c4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.710711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96421ec8-a99b-4202-b853-50ca94518962 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.718887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.743479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.744216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.744488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.744761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.745414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:55.745713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.747409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.751810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.752490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.752769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.753039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.759670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 203/405] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:55.760786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.770532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=b45a976b6672412f92f9a4f8fbc75b1b, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.772072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.787258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 203/406] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:55 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:55.788178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.888467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2b488-c8f7-4132-b067-afa559570251 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.898161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:55.926301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f05bc-c85e-4ecf-8b79-fbc11fb9a58e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:55.933366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.934007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.934287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.934563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:55.935402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:55.935720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:55.937790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:55.959265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 204/407] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:55.959265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:55.984625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:55.985277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:55.985551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:55.985818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.002142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.003960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.005925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:56.021713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:56.022842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:56.024295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:56.024295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:56.024295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:56.024295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 204/408] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:56.024295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.087765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad9e31-1a61-4632-9dde-7788361c1ea8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.122947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.123624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.123939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.124292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.131337 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.132913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.144131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94475a10-35bf-48ad-8093-debcd89ed355 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:11:56.144792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:56.145778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:56.146809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 205/409] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:56.147573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.187440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.188110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.188390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.188664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.190922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.197215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.199628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:11:56.213978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:56.215038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:56.216031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 205/410] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:56.221740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.259417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.270690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb3a1f-cfc9-4c6f-8722-4206e0378dbf None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.312176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.313131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.313553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.313953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.314891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:56.315631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.317047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.335778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 206/411] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:56.336919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.341849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9efea-af52-4222-81e8-f8a332d5ecd7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.377767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.378629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.379046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.379515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.380437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:56.380878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.382461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.404423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 206/412] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:56.405594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.463680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07ba04-3d90-4286-8408-642f81b9b503 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.499540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.507196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.508419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.508836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.509268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.510104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:56.510971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.513067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.523359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194b793-0f1b-492f-9831-2aecc12a6d1f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.528797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 207/413] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:56.529876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.556878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.571022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.571732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.572042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.572447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.581001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.582671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:11:56.595894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:56.596948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 207/414] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:56.598076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.679078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35ec55-f471-4460-8128-0c9db9c1b181 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.711541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e98822-f09c-4fb5-a4d4-d79a4d1a9f1f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:56.724550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.726168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.727087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.727902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.728446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/7623623b296e4d04949973891b697c62/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.750326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=7623623b296e4d04949973891b697c62, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.751759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.753007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:56.754221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:56.755183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:56.755862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/7e49839dc5bc47d6a9a91800fef249c1/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:56.773587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 208/415] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:56 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/7623623b296e4d04949973891b697c62/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:56.774959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=7e49839dc5bc47d6a9a91800fef249c1, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:56.775398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:56.776572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:56.798082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 208/416] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:56 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/7e49839dc5bc47d6a9a91800fef249c1/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:56.889449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:56.945380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a36f5-1d0b-4399-857b-06f03e4798b5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:57.000677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:57.001748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:57.002169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:57.002575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:57.003470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:57.003912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:57.005522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e171ff-48fe-4299-b780-48801862d74d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:57.023433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 209/417] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:57.032605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:57.149700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41af19b-18fe-4d4d-be49-a9e77dc3fec7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:57.190684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:57.191417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:57.191705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:57.191978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/cd98a9aa326741fba8ff2360f8e87e0e/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:57.214483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=cd98a9aa326741fba8ff2360f8e87e0e, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:57.216036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfc992-9268-4d28-a9c1-93b064092841 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:57.241151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 209/418] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:11:57 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/cd98a9aa326741fba8ff2360f8e87e0e/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:11:57.979995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1f8db-48a7-478b-ae4d-714fb2bac970 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:57.980188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1f8db-48a7-478b-ae4d-714fb2bac970 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:57.980478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1f8db-48a7-478b-ae4d-714fb2bac970 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:57.982069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 210/419] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:57.987793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131498-a5a1-41ca-b5c9-ab7f68c4f889 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:57.988084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131498-a5a1-41ca-b5c9-ab7f68c4f889 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:57.988365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131498-a5a1-41ca-b5c9-ab7f68c4f889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:58.007263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-25131498-a5a1-41ca-b5c9-ab7f68c4f889 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:58.008470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-25131498-a5a1-41ca-b5c9-ab7f68c4f889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:58.018871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:58.088803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 210/420] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:11:58.363117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26486a-09a6-4d53-8cf4-a2d9db0bb921 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:58.363687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26486a-09a6-4d53-8cf4-a2d9db0bb921 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:58.363687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26486a-09a6-4d53-8cf4-a2d9db0bb921 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:58.365401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 211/421] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:58.372883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:58.373338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:58.373434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:58.392282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:58.393478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:58.432054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26486a-09a6-4d53-8cf4-a2d9db0bb921 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:58.497954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 211/422] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:58.560982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26486a-09a6-4d53-8cf4-a2d9db0bb921 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:58.593976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:58.617037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:58.617621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:58.617905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:58.618176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:58.639108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:58.644106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:58.644985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:58.646379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 212/423] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:58.651072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:58.745158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:58.777167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cccc71-f4d7-401b-b112-4491fd7c8779 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:58.818656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:58.819304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:58.819581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:58.819847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:58.820522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:58.820817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:58.822202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:58.835283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 212/424] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:58.836844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:58.841305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:58.970449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e41423-c07c-4512-90e7-e5a683b93a99 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:58.988459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d544b5-2b1a-46d7-8511-8ed17324f29f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.006603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.007685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.008154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.008628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.020599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.022477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.024217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.025046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.025326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.025591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.028808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.039045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:59.041382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:59.042255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:59.043085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:59.043085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:59.043085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:11:59.044369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:59.045198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:59.050296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 213/425] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:11:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:11:59.053222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 213/426] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:59.054691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.058312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.127585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.213111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855cb5-ed75-42f1-bf4e-31121368016c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.226237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975825a9-ffd7-435b-85bf-397bb42cb4b0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.271876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.273272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.274065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.275044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.275860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.276764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:59.277565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.278110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.279074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.279550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.280795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.281631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:59.282174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.284960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.301109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 214/427] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:59.308070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.313219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 214/428] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:11:59.323340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.388229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.480833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1588ac-19dc-4ccb-be3f-e663eccca7c1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.507697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.519170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b2d63-1c8a-4dc7-9c43-83c70e5ca5db None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.546723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.547393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.547673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.547942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.559359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.560850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.562817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.563482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.563766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.564037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.571200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.572655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3e89ab72-9821-4fef-923f-4138cf6af94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:11:59.575200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:59.576113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:11:59.576956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:11:59.576956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:11:59.576956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:11:59.576956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 215/429] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:59.577651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d074ec-1d26-46ad-a4b5-1e199b2648f9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.577932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d074ec-1d26-46ad-a4b5-1e199b2648f9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.578202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d074ec-1d26-46ad-a4b5-1e199b2648f9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.580116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 216/430] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:59.581375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b34b5-8931-46f7-ae06-c74578a24140 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.581498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b34b5-8931-46f7-ae06-c74578a24140 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.581757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b34b5-8931-46f7-ae06-c74578a24140 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.586894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 217/431] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:11:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:11:59.586894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b34b5-8931-46f7-ae06-c74578a24140 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8eec37de-b117-4026-8363-45189a960d9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:11:59.592700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:11:59.593669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:11:59.594555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 215/432] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:11:59.597394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27978479-8a89-4a3b-9dc7-694d64bd626a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.597869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27978479-8a89-4a3b-9dc7-694d64bd626a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.598731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27978479-8a89-4a3b-9dc7-694d64bd626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.622323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-27978479-8a89-4a3b-9dc7-694d64bd626a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:59.625522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-27978479-8a89-4a3b-9dc7-694d64bd626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:59.697697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 216/433] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:59.698961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.699273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.699544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.708040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b34b5-8931-46f7-ae06-c74578a24140 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.722419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:11:59.723763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:11:59.724348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.750888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:11:59.752042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:11:59.752483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:11:59.752759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:11:59.753416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:11:59.753719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:11:59.755189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:11:59.770372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 218/434] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:11:59.775604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.800608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 217/435] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:11:59.801786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:11:59.834251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:11:59.924316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd874b5-dd8a-4e31-abb0-009a6a86f1d1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:11:59.971557 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b589817-3a66-469b-9aae-b771067b53e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.005546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.006116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.006396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.006666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.007509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:00.007757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.009209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.010430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.011454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.012712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.013090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.022076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.023687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.028369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 219/436] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:00.029285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:00.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:00.039937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:00.040858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:00.041820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 218/437] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:11:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:00.042491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.065348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.127309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.147044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d1cc-c39e-4a17-9ce7-387117a1eb36 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.153211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2bdef-4b78-4664-af34-63911a9aeb92 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.180779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.181412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.181688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.181952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.185603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.186229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.186542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.186832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.191967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:00.196822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:00.197691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:00.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:00.199099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.199613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 220/438] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:00.199613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.199775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-43c3dea3-ec52-4a2d-9b74-af8e7e01ca3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:00.214170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:00.215109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:00.215987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:00.215987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:00.215987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:00.215987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 219/439] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:00.216710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c30ea-b439-4924-8b2c-755ac3bfb513 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.216994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c30ea-b439-4924-8b2c-755ac3bfb513 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.217268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c30ea-b439-4924-8b2c-755ac3bfb513 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.218519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 220/440] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:00.219231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c30ea-b439-4924-8b2c-755ac3bfb513 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.292708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.305204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40f0fe-299f-481d-91a3-7373446044f5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.317480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c30ea-b439-4924-8b2c-755ac3bfb513 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.338914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.339579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.339856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.340123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.340851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:00.341149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.341690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.342608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.350604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.351247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.351522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.351788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.364110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 221/441] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:00.364110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.364327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:00.367037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:00.368212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 221/442] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:00.369166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.463634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e928f-6bed-4a35-b057-0d70975a007e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.466916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42660d4d-ecb1-4dbb-b747-cbd895fde2e8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.499420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.500033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.500313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.500403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.500586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.500947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.501222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.501306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:00.501486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.501603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.502121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:00.502416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.503024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee82fc-bf81-4668-9367-802f74137288 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.503813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.511136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 222/443] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:00.512304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.512590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.512862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.516034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 222/444] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:00.516687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.531436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:00.532780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:00.534716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.566544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.602060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 223/445] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:00.602896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.626698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb1696-caf2-47e9-b08c-1795c59bd3b1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.663224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.663925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.664210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.664483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.710973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58093084-0eb4-4087-9318-ca6b3dc56e50 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.732900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.734530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.745305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.745917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.746196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.746463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.747123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:00.747595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.748973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.754938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.763058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 224/446] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:00.763058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.787431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.845140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 223/447] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:12:00.846544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.868474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fdd58-5f72-4082-82f1-d6a4a0ca8dfc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.901025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.901653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.901922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.902182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.912223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.913660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:00.927851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:00.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:00.929827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:00.929827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:00.929827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:00.929827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 225/448] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:00.929827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:00.948949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f6456-1064-4c5d-a5d4-c65954fc0dcc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:00.980532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:00.984666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:00.985297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:00.985572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:00.985842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:00.992502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:00.993953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:01.005098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.006088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 224/449] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.007297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.015438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.031526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0cdb9-e2b3-4fb8-b097-5c7622143846 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.066865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.067517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.067793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.068064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.078713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.080595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:01.094481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.095634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:01.096536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:01.096536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:01.096536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:01.096536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 226/450] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.096536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.107046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269c295-9cb1-4478-8704-0206a2c1dde2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.139138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.139836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.140113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.140409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.150508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.151997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:01.165729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.166737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:01.167818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:01.167818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:01.167818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:01.167818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 225/451] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.167818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.199843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e710234-a4a1-4d03-a9d2-9ef93b6fd3e9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.202101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.227384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.234512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.235173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.235449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.235716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.245539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:01.250831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.251944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:01.252767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 227/452] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.253444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.274748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b5ca7-2c41-4aa6-a0d4-4bfd9a50858e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.311485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.312116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.312384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.312657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.313325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:01.313625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.315019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.326670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 226/453] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:01.327520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.357606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6084c8f-44e6-4cf3-9c0b-45c949b915c1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.390343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.390976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.391279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.391546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.392238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:01.392537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.393967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.405796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 228/454] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:01.406752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.426833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefee51-22f8-438b-beb4-16053c44b7fa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.432791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.444142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.461051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.461653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.461925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.462188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.462824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:01.463120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.464510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.479885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 227/455] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:01.479885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.507106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb481d77-a1c1-45f3-a931-8f6ce64eda04 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.540462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.541323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.541587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.542226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:01.542538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.543938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.554836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 229/456] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:01.555705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.574959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b757a3-deed-4013-9d18-6816a8ccea9c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.608837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.609378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.609654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.609923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.610588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:01.610894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.612353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.620360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 228/457] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:01.621134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.644041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.660261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eb82d-91b7-4a78-8c9f-4019ad1ae607 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.663321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.693078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.693713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.693990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.694267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.701671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.703192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:01.715567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.716667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 230/458] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.722549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.723351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7e749-5d4a-4266-8dcb-292127cba9bd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.753002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.753651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.753920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.754189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/084038e0639b49c99485f53463392d11/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.768047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=084038e0639b49c99485f53463392d11, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.769513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.782485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 229/459] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:01 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/084038e0639b49c99485f53463392d11/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:01.783401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.828913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5513db-8894-42f4-a64d-7fcfcc277213 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.871071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.872402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.872775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.873157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.883792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.886021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.888736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:01.899282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.900458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:12:01.901589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:01.902220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa283a3-4eb1-4b85-b6f8-b1074caba1c1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:01.903293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 231/460] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.904096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:01.909165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:01.945573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:01.946382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:01.946763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:01.947208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:01.955071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:01.957455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:01.971416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:01.972523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:01.973563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:01.973986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 230/461] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:01.974713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.019270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d638be0-d246-4ff6-b04a-81d8c05c4b8c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.055239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.055868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.056140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.056406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.067821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.069354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:02.084681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:02.085750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:02.091441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:02.091441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:02.091441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:02.091441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 232/462] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:02.091717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb70a0-8a53-4a97-be0c-b91f4b8a209a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.091868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.125724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.126355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.126714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.127047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.127882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:02.128249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.129791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.147234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 231/463] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Oct 25 13:12:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:02.152329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.153883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.163322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.203366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf2d0-2c4a-4d06-a084-ae27dc9f3f43 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.238468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.239289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.239659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.240091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.240879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:02.241232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.242858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.254382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca84a82-8f5f-467a-8035-ddf1373ef130 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.266950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 233/464] 10.222.0.36 () {56 vars in 1137 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:02.266950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.293996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.294620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.294901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.295199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.295954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:02.296260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.297705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.315135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 232/465] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:02.315135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.378606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.381473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844be65-e252-4786-9b35-eba54c2bd5bf None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.395800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.416921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29a319-2f4e-43d5-886f-f27e7986ae15 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.418157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.418875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.419290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.419674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.420429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:02.420861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.422355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.438009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 234/466] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:02.438009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.456371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.456983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.457257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.457521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.527440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.529086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.540556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e58dc-07b1-401d-b5aa-1c031398a8b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.575685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.576533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.576807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.577115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.600731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.628583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.644917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 233/467] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:12:02.650012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.655991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.657609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.750669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b727aa4-0a89-450d-95c6-f30eb931155b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.772693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 235/468] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:12:02.777838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.784767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.785444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.785785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.786127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.786919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=b45a976b6672412f92f9a4f8fbc75b1b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:02.787325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.788839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.799181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 234/469] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/role_assignments?user.id=b45a976b6672412f92f9a4f8fbc75b1b&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:02.799181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.846237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.877723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:02.905365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913da4c-fa3f-466b-853f-3fa4b7aded26 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.942429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.943194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.943555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.943884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:02.946391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c57-0081-4d45-a5ad-47e4af18dd79 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:02.951993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:02.953743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:02.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:02.968338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 25 13:12:02.969464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:02.974804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 236/470] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:02.975820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:02.997358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:02.998718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:02.999563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:02.999843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.015384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.016899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:03.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:03.033999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:03.034892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:03.034892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.034892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:03.034892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 235/471] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:03.035595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.088589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac95a63-0f21-4b43-9957-b3556fc4d48f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.127615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.128297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.128571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.128878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.131083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.143250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.144075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.145141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:03.163342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:03.164459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:03.165502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:03.165502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.165502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:03.165502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 237/472] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:03.165722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.166082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85e648-dc12-4ea1-a7ed-602f62bafc2a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.205236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.205844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.206117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.206382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.207113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:03.207582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.209175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.228089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 236/473] 10.222.0.36 () {56 vars in 1139 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:03.229159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.286543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ed120-cd81-4c77-bfaa-cd00c5dc6867 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.327166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.328127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.328675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.329110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.330280 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:03.330731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.332915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.343469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed186399-9b3f-4359-a3c5-eb3f95e20d26 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.349027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 238/474] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:03.350164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.380266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.380969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.381246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.381559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.382207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:03.382530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.384143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.396845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 237/475] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:03.397615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.408585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.411507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.474934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05fc518-bf64-47b9-8390-9401ac458d5a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.512637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6a9e9-6311-4419-8d81-d4ec32cc6542 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.514114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.514869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.515132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.515428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.550151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.551100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.551597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.551997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.559531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.561388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:03.574421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:03.575510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:03.576524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:03.578656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 238/476] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:03.580159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.594343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.595957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.649169 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.661166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.705859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5a7da-4597-4ad8-a5e9-b8d5222b8c5e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.711292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 239/477] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:12:03.712225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.743365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.743974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.744251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.744558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.751691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.753146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:03.765478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:03.766367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:03.767258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 239/478] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:03.768169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.821873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91850b-385d-4373-9189-9be61a42d202 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.858810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.859771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.860211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.860745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.861710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:03.862156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.863853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.878042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 240/479] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:03.879380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.880308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12b08-664c-45a8-ac52-9dea56c74fab None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:03.889382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.908610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:03.922213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:03.923227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:03.923751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:03.924284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:03.936160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:03.938017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:03.953630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:03.954915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:03.955810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:03.955810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:03.955810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:03.957728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 240/480] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:03.958842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:03.986412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2189f-f92c-4954-b16b-1819489ae61c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.021085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.021702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.021975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.022254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.022963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:04.023273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.024719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.036577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 241/481] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:04.037442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.069518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3617c-baae-4dce-99e2-776daca11c55 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.104930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.105775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.106189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.106593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.107511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=7e49839dc5bc47d6a9a91800fef249c1) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:04.108038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.108516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:04.109801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45560f-444d-42b6-bc46-6242815a445a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 241/482] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:03 2022] GET /identity/v3/role_assignments?user.id=7e49839dc5bc47d6a9a91800fef249c1&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492032-eef9-48ef-861f-e92e5a90008d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492032-eef9-48ef-861f-e92e5a90008d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39492032-eef9-48ef-861f-e92e5a90008d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 242/483] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:04.126060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492032-eef9-48ef-861f-e92e5a90008d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.157411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410b8a-eda9-411f-8b25-bcd69d948460 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.181910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:04.204029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.205575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.206014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.206430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.207950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:04.208430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.210503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59c733-326c-4abd-a4f9-3dcb86381dda None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.227503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 242/484] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:04.229654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.230472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.230916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.256643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:04.259443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:04.263468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39492032-eef9-48ef-861f-e92e5a90008d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.312421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.313161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.314238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.314977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.315472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=7623623b296e4d04949973891b697c62) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:04.315880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.317482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.324382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 243/485] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/role_assignments?user.id=7623623b296e4d04949973891b697c62&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:04.325177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.353322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 243/486] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:04.470381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11220bf4-ba00-40a3-9ca2-896fd6e1435c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.472281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:04.511233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.511885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.512166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.512438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.520532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.522107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:04.534822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:04.535766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:04.537028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 244/487] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:04.542222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.618692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:04.682019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561223a5-aab6-4c8d-96ec-2c973ffb9abe None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.691766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.741823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.742599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.744120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.744488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.745296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:04.745889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:04.748060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:04.774473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 244/488] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:04.780511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.889854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b586-2fe9-4cac-ade4-d704b6aa79e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:04.945918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:04.946771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:04.947156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:04.947527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:04.958534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:04.964119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:04.965233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:04.966235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:04.968283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 245/489] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:04.974466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:04.991446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19c981-0020-4991-8471-97112e6afd08 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.024905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:05.028473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:05.047742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.048697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.049176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.049649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.050761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=cd98a9aa326741fba8ff2360f8e87e0e) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:05.051337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:05.053019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c15d4-0840-46db-9623-5d4adc13d5c4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:05.061148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 245/490] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/role_assignments?user.id=cd98a9aa326741fba8ff2360f8e87e0e&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:05.163352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b6e6-f90b-42f4-a29e-9c79bc10ee52 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.196492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90041bf-a443-4e35-a418-d12a7e18f1e6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.197180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90041bf-a443-4e35-a418-d12a7e18f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.197674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90041bf-a443-4e35-a418-d12a7e18f1e6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.199328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 246/491] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:05.212905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.213224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.213452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.220914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.221896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.222400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.222864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.223912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:05.224537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:05.226254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:05.237091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 246/492] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:05.243842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:05.256388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:05.257612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:05.354592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:05.355800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 247/493] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:05.372136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8f2a1-1de2-49ff-9117-6992e3cdc0f0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.412787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.413472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.413701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.413963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.424750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:05.426203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:05.441444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:05.442366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:05.444003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:05.444003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:05.444003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:05.444003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 247/494] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:05.450494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:05.565289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9e64d-81b8-4c57-be14-ac02d40d809e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.576223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:05.604475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.605095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.605374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.605647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.606359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:05.606669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:05.608072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:05.624490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 248/495] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:05.630811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:05.698956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:05.759538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f484099-25b5-4545-91e4-69ddd4f297ff None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.825823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.826460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.826786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.827087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.839701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:05.849080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:05.850548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:05.870698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:05.871724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:05.872845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 248/496] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:05.879236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68f71-7b42-469d-867a-691a661aa084 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:05.881821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:05.940247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:05.940882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:05.941159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:05.941423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:05.963733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:05.975241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:05.976201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:05.977048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:05.978164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 249/497] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:05.984385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.043934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.064884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11023-d90f-4d9c-9469-a0234807c65b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.115414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cdbb4-5909-4f34-83a7-711f2ca67bcd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.127526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.128559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.129080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.129660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.130590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:06.131086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.132920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.155077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.155759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.156037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.156305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.156983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:06.157283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.161196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.170509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 249/498] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:06.174144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 250/499] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:06.179931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.185173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.252637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.300612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9392c8-05a6-43cf-bc86-bd08d74d8939 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.337692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.338692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.339201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.339686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.351294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.353226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.366445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:06.370666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:06.371623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:06.373027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:06.373027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:06.373027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:06.375354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 250/500] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:06.379251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169522f-e740-4d6e-bb7b-a1ea50d3dbfe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.381615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.414070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.414688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.414934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.415227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.487914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.489566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.515194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581606d9-0277-4fb1-809f-b87baffbfbc5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.522530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.551776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.552768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.553276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.553739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.554590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:06.555085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.556700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.569413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 251/501] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:06.575022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.583014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.594720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 251/502] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 25 13:12:06.623936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.691067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bb9a9-1811-4521-b5b2-9c44767250ff None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.730400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.731059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.731369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.731639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.745996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.747508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:06.783267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:06.784685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:06.785662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:06.799177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 252/503] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:06.799177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:06.829202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a531b-9466-4c4b-aaff-3c26e2686874 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.832715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:06.919681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f606078-db68-42b7-9cc1-e8b34d814bba None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:06.921860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.922845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.923371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.923842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.948468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.950273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.974071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:06.974688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.974967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.975336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.976083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:06.976404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:06.977839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176c214-c5c7-4deb-96b4-b30871df4ef8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a8990297-1fcf-4630-9b43-9c5273c46f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:06.979242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:06.980164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:06.981095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:06.981095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:06.981095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:06.984025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 252/504] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:06.985846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a7c8-aa5e-4257-aca9-22d725171ec8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:06.986335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a7c8-aa5e-4257-aca9-22d725171ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:06.986838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a7c8-aa5e-4257-aca9-22d725171ec8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:06.992684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 253/505] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:06.994428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555a7c8-aa5e-4257-aca9-22d725171ec8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.020158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 253/506] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:07.021471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8af8-dcb8-4ac0-a57a-d7db85e87c2e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.021761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8af8-dcb8-4ac0-a57a-d7db85e87c2e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.022033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d8af8-dcb8-4ac0-a57a-d7db85e87c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.040999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5e1d8af8-dcb8-4ac0-a57a-d7db85e87c2e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:07.042127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5e1d8af8-dcb8-4ac0-a57a-d7db85e87c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:07.047279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.115818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555a7c8-aa5e-4257-aca9-22d725171ec8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.131739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 254/507] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:07.133520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7497-5696-4c56-8a40-60e322a6a1f1 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.133815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7497-5696-4c56-8a40-60e322a6a1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.134094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7497-5696-4c56-8a40-60e322a6a1f1 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.135677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 255/508] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:07.136390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7497-5696-4c56-8a40-60e322a6a1f1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.155953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.156576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.156849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.157172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.158384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:07.158686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.160168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7a93f-12d5-4a07-b101-fbf4d0de4fc1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.177888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 254/509] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:07.179230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.179516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.179783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.192292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.199452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:07.200835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:07.303322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 255/510] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:07.303322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.335856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.338333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7497-5696-4c56-8a40-60e322a6a1f1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.412570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddec5e3-23ce-4889-bc1b-124fbd6bc9ca None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.416391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.417143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.417536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.418242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.433891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.435547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:07.450908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:07.452125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:07.453219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:07.453219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:07.453219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:07.454888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 256/511] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:07.455612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.460556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.460556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.460556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.460556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.468780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.470244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:07.482122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:07.483076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:07.484049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 256/512] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:07.484548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.546749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.582614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.590067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c35b-42a7-4d05-8ad1-681acad28242 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.596295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485952e-64b0-42a3-a17a-bcb63e373f8f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.635181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.635822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.636097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.636368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.637023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:07.637327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.638781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.645357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.645998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.646275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.647026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.647735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.649554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.652778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 257/513] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:07.653247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:07.662171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:07.663248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:07.664715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 257/514] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:07.665532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.756743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facc1c5-98f6-4449-a6ea-ae1e08832503 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.769341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a29d-4c9b-41f6-aab6-0057b5aea07a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.794438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959d500-3079-4f33-a096-0498805d193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.795064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d500-3079-4f33-a096-0498805d193f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.795657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d500-3079-4f33-a096-0498805d193f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.795657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959d500-3079-4f33-a096-0498805d193f None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.796475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959d500-3079-4f33-a096-0498805d193f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:07.796748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959d500-3079-4f33-a096-0498805d193f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.798255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959d500-3079-4f33-a096-0498805d193f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.802092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.809416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.810066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.810347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.810618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.811751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 258/515] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:07.816123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959d500-3079-4f33-a096-0498805d193f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.822451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.824006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.835653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:07.841365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:07.842298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:07.844137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:07.844137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:07.844137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:07.844137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 258/516] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:07.844137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.927478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959d500-3079-4f33-a096-0498805d193f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.947642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a784-7c8e-49e9-8ce9-6a497fc497f0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:07.960793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.961429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.961708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.961974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.968534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.969993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:07.981060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:07.981986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:07.982911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:07.983398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:07.983398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:07.983398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:07.983631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 259/517] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:07.983712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:07.983712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:07.983847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:07.983914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:07.984654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:07.989340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 259/518] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:07.990125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.038885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.058816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.083231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaa763-a46f-4882-8cac-c4774ac4482b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.092107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc40965-fe97-417e-a728-05e2b168cd2c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.115056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.115692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.115972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.116243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.116986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=084038e0639b49c99485f53463392d11) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:08.117300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.118744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9e2f5-f53c-4a2f-82c3-200d9db0b3ae None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.124476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 260/519] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/role_assignments?user.id=084038e0639b49c99485f53463392d11&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:08.125654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e741-3f9b-4206-a76b-11b7cfaf3cd1 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.125936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e741-3f9b-4206-a76b-11b7cfaf3cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.126206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e741-3f9b-4206-a76b-11b7cfaf3cd1 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.127439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 261/520] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:08.128159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e741-3f9b-4206-a76b-11b7cfaf3cd1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.134234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.134871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.135168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.135470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.136141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:08.136448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.137904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.154118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 260/521] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:08.155420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.258308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e741-3f9b-4206-a76b-11b7cfaf3cd1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.266075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dece34-a16b-41c9-804f-5c3f55485d78 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.275199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.300114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.300761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.301015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.301278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.301986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:08.302292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.303926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.304949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.306679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.306679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.306865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.307525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:08.307823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.309185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbdb8c-5ad0-4c3b-b5d4-901b5f50ac4e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.312455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.315224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 261/522] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:08.316540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.316979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.317382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.319323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 262/523] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:08.320216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.341147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:08.342445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:08.433117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 262/524] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:08.434437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.466611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da284f-42eb-43e7-b8a6-136ed470297b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.537110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.538005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.538285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.538558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.539279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.541049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.568306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 263/525] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:12:08 2022] POST /identity/v3/users => generated 254 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:08.594358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.597515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88895158-9e2e-43b6-9c41-c946d51dc511 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.638561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.639285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.639628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.639899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.642167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=b45a976b6672412f92f9a4f8fbc75b1b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.643836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.663957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 263/526] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:08 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:08.717939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:08.796908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:08.856562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a9c6b-a972-4668-9085-465d7d40d268 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:08.900992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:08.901650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:08.901926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:08.902190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] PATH_INFO: `/v3/services/network` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:08.902801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:08.904281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:08.916495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:08.917565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:08.918423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 264/527] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:08 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:08.924201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:09.033525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2e64-570d-408e-8e21-ddf3398fca20 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:09.075919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:09.076563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:09.076839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:09.077173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:09.077896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:09.078204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:09.079996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d2197-5e51-4afc-8528-88b3e753b2d7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:09.085282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 264/528] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:09.090663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:09.245343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4327-8619-4c9b-9922-0e02c1dcf5ba None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:09.269567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:09.292892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:09.293511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:09.293788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:09.294053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:09.294690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:09.294988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:09.296455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3998f-4381-4a55-a100-4f2939f2579f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:09.306258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 265/529] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:10.648368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4651e-7202-4b6c-b175-0c984086f6f8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:10.648791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4651e-7202-4b6c-b175-0c984086f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:10.649009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4651e-7202-4b6c-b175-0c984086f6f8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:10.650651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 265/530] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:10.655626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1113-c98e-414d-88a7-e4590e244e55 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:10.656036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1113-c98e-414d-88a7-e4590e244e55 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:10.656201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1113-c98e-414d-88a7-e4590e244e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:10.675683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-da5d1113-c98e-414d-88a7-e4590e244e55 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:10.676871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-da5d1113-c98e-414d-88a7-e4590e244e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:10.753703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 266/531] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:10.910177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acbdba-a869-47df-b75b-84ff6f03464a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:10.910354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acbdba-a869-47df-b75b-84ff6f03464a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:10.910630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acbdba-a869-47df-b75b-84ff6f03464a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:10.912010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 266/532] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:10.917548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:10.917839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:10.918119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:10.936659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:10.937856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:11.025715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 267/533] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 25 13:12:11.114429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acbdba-a869-47df-b75b-84ff6f03464a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:11.148875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:11.254925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acbdba-a869-47df-b75b-84ff6f03464a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:11.302196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:11.303214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.303692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.304158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.321888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:11.327289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:11.328223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:11.332413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:11.336183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 267/534] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:11.346252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:11.347845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:11.429057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:11.516838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:11.535188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200bb0c-205d-4a96-ae63-1628b77a49cd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:11.542517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95735d5a-0e1e-4f4b-9703-b0455c58e9ce None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:11.576438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:11.577138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.577419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.577693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.578392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:11.578695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:11.580528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:11.603753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 268/535] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:11.610336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:11.615347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:11.616153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.616575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.617014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.617816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:11.619423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794ee9-d278-4323-a935-81096a66d15e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:11.640168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 268/536] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:11 2022] POST /identity/v3/services => generated 251 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:11.641746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002489e-60b8-466c-959b-747abb839405 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.642187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002489e-60b8-466c-959b-747abb839405 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.642548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002489e-60b8-466c-959b-747abb839405 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.644449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 269/537] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:11.649710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77453b-8637-41b2-b7f2-989ab9b39b63 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.650333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77453b-8637-41b2-b7f2-989ab9b39b63 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.650427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77453b-8637-41b2-b7f2-989ab9b39b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.701904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc77453b-8637-41b2-b7f2-989ab9b39b63 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:11.703215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc77453b-8637-41b2-b7f2-989ab9b39b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:11.738052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317ed07-b3b4-472b-a3db-7a032f62f88d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:11.741291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:11.791447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 270/538] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:12:11.793255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6c27-3fab-4ca0-8d49-5204442afb0c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.793800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6c27-3fab-4ca0-8d49-5204442afb0c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.794213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb6c27-3fab-4ca0-8d49-5204442afb0c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.799233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 271/539] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:11.803074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:11.803074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.803074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.803074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.805293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e3e5-9c1c-41ff-b4f6-1a16fdadb031 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.805576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e3e5-9c1c-41ff-b4f6-1a16fdadb031 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.805849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e3e5-9c1c-41ff-b4f6-1a16fdadb031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.833355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:11.835172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:11.838702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-91f8e3e5-9c1c-41ff-b4f6-1a16fdadb031 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:11.840068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-91f8e3e5-9c1c-41ff-b4f6-1a16fdadb031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:11.862855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:11.863921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:11.864808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:11.864808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:11.864808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:11.870323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 269/540] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:11.871623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:11.926917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:11.933199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 272/541] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:12:11.937501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa323-beec-4786-ab02-776116fce106 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.938044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa323-beec-4786-ab02-776116fce106 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.938479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa323-beec-4786-ab02-776116fce106 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.940722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 273/542] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:11.946365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:11.946647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:11.946922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:11.965870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:11.967239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:12.002985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5e33-4cf1-4edf-b7f0-4dc88d3dde5f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.047988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.048526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.048839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.049111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.049752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:12.050046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.050988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 274/543] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 25 13:12:12.051461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.065388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 270/544] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:12.070569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.101792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.117722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.214466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4705c7-5c1b-4d6e-9fad-775ea67c82a0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.235240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cd316-b4d9-44aa-8f3a-6100c5861fc9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.267007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.276084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.277023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.277303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.277571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.284254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.285129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.285595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.286067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.286181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.287545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.297429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:12.301530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:12.302436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 275/545] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:12.303931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:12.305830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:12.306705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:12.307871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:12.314700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 271/546] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:12.315901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.389999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.415404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5bd86-3935-4311-b703-288d2a4cc375 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.425493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8736e-55c1-49cb-9c46-28d332dbe433 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.455484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.456333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.456474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.456675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.461332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.461941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.462217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.462509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.463235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:12.463539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.464972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.467420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:12.473533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:12.474484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:12.475416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 276/547] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:12.477640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.487888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 272/548] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:12.488708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.532996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.585091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de003299-1c7b-4575-9f68-f0ac59e192b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.590639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51f47f-fb45-49f0-a828-bae7fd54bfb0 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.622767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.623759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.624416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.624724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.624945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.625625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:12.625926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.627329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.633578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.634194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.634471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.634739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.635420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.636790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.641880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 277/549] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:12.641880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.663688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 273/550] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:12 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:12.664709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.771796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fc3fc-94ce-4f1e-8bfb-93a38682f905 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.787561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.796510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35b95-4d36-4d8e-9b8c-466ad209e055 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.813534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.814650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.815134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.815985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.817042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:12.817571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.819314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.835685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 278/551] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:12.837061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.840428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:12.840932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:12.841241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:12.841514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:12.854558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:12.856080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:12.872193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:12.873189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:12.874156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:12.874156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:12.874156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:12.874156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 274/552] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:12.874946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:12.921339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:12.956386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f61af-196b-4792-9eff-87d37a95185b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:12.999861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10549-52ef-43ea-bf18-30b3cffda528 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.001918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.003389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.003680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.003956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.020329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.022504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.040704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.041341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.041617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.041893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:13.043372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:13.044311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:13.045214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:13.045214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:13.045214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:13.047784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 279/553] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:13.048938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.056971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.058619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:13.076715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:13.077957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:13.079076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:13.079076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:13.079076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:13.081407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 275/554] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:13.082545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.087677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.169145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e79d7e-ea1d-4934-893c-7118b21e46fb None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.202362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.206301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.207027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.207241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.207514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.208193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.208492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.210918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.215935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ce8d-991d-45e8-86a2-86eec55e6e1b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.224373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 280/555] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.225724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.262522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.263695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.264117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.264945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.266075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.266522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.268341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.284682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 276/556] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.285794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.328019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.346245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe783d4-7d0a-4f27-9327-18c667fd6006 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.386882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.387659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.388029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.388391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.389135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.389823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.391350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.404291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 281/557] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.405494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.420537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dbe3c-3efb-42b0-8369-aec87a0367e3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.464002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.464933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.465517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.465928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.475044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.477336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:13.491861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:13.492970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:13.494038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:13.496888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 277/558] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:13.498024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.514474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.516311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd408595-5834-4253-bf81-a198b8bea149 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.557738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12030-407f-4372-8470-61900041229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.559072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12030-407f-4372-8470-61900041229d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.559522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12030-407f-4372-8470-61900041229d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.560189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c12030-407f-4372-8470-61900041229d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.570163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12030-407f-4372-8470-61900041229d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.572020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c12030-407f-4372-8470-61900041229d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-58c12030-407f-4372-8470-61900041229d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:13.587545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:13.588730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:13.589893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:13.592258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 282/559] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:13.593276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12030-407f-4372-8470-61900041229d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.603714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.629373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22da9a-03a2-4203-a578-74f766e94253 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.667491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.668285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.668686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.669046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.677920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.679548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:13.691606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:13.692782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:13.693938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:13.695737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 278/560] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:13.696837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.727232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c12030-407f-4372-8470-61900041229d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.769195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.770069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.770726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.771141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.772029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.772558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.773144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.775784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.790880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 283/561] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.792261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.807137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b5de7-2d21-48ac-8836-16a99072f8e2 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.845477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.846349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.846924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.847366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.848643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.849224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.851121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.867481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:13.874107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 279/562] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.875194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:13.910788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551ed3-a65c-4cc1-92ff-6d15e1195348 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:13.947428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:13.948310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:13.948687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:13.949052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:13.949935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:13.950336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:13.952159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:13.971536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 284/563] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:13.972592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.000589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898864-8e7d-40fd-93bb-02d6fedc3a77 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.037637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.047887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.049216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.049769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.050199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/084038e0639b49c99485f53463392d11/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.052460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=084038e0639b49c99485f53463392d11, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.054837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.073855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 280/564] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:13 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/084038e0639b49c99485f53463392d11/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:14.075100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.091173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dc8f6-2105-494d-9ce3-39a3e652ec96 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.119208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.134554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.135388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.135771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.136139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.213955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14123-3dd1-4a22-99eb-e1836f9064fe None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.218471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.220278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff48c3-00db-4d6f-a557-1de473907d8c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.263469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.265062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.265491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.265913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.280310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.282542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-084a9972-94ea-48f5-98d8-7f77ebad6aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:14.303132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:14.304300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:14.305380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:14.305380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:14.305380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:14.307219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 281/565] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:14.309353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6444d3-1da1-4314-8fa4-5f8f2835dcef None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.309805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6444d3-1da1-4314-8fa4-5f8f2835dcef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.310503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6444d3-1da1-4314-8fa4-5f8f2835dcef None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.312481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 282/566] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:14.313890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6444d3-1da1-4314-8fa4-5f8f2835dcef None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.333006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 285/567] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:12:14.334544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.334868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.335216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.368175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:14.369595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:14.373078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.399544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.448374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6444d3-1da1-4314-8fa4-5f8f2835dcef None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.455185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 286/568] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:14.455185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.516044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.516663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.516938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.517206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.517859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:14.518157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.519581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.543657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 283/569] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:14.549570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.593500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c872e60-86f2-4b92-82d0-89d00af9a865 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.642896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.643581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.643858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.644128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.655564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.657111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.671123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.674006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621dd37-e3e3-4ea9-8235-8ad9ce43f541 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:14.674245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:14.675466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:14.676470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:14.676470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:14.676470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:14.680618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 287/570] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:14.682622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.692697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.715370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.716334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.716846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.717313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.724424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.726124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:14.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:14.738607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:14.739561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:14.742826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 284/571] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:14.748691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.793190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29198d91-a65a-4e62-bcdb-963c7b28ce97 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.833432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.834050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.835002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.835523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.837337 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:14.837641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.839023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.862720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 288/572] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:14.863788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.887418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88b90-cb6f-48fc-ae67-90469336ef03 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:14.925767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:14.926395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:14.926673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:14.926943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:14.927699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:14.927999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:14.929551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:14.941864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 285/573] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:14.942697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:14.943313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.965941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:14.986604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd44b1-5adb-4c8e-970a-aa78f1355eea None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.024714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.025353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.025937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.026215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.026970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.028390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:15.037455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:15.038442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:15.039317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 289/574] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:15.040109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.051545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca99f2-0af5-443e-88a9-48a16b287f07 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.087786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.088431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.089082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.089354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.096393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.097841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-deb7e2af-eed3-4e9d-b3dc-3b26e0c41355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:15.109920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:15.111226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:15.112212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 286/575] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:15.112865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44897-9d0e-4c71-aed9-b203bc65e819 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.113152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44897-9d0e-4c71-aed9-b203bc65e819 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.113423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee44897-9d0e-4c71-aed9-b203bc65e819 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.118874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 287/576] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:15.118874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee44897-9d0e-4c71-aed9-b203bc65e819 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.151100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341185-5a26-49b9-8f7d-d698f8a132dd None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.183366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:15.190560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.191799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.192125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.192403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/5afba0fee3f74a5883257487f6c5c84b/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.206258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:15.209289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=5afba0fee3f74a5883257487f6c5c84b, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.210816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.223848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 290/577] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:15 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/5afba0fee3f74a5883257487f6c5c84b/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:15.231399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.237203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee44897-9d0e-4c71-aed9-b203bc65e819 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.291580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.292585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.293390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.293675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.294607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:15.294914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.296692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71e2c-a970-44b8-a03a-072c52dfd8f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.307462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 288/578] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:15.310300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.310807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.311357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.338950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:15.340812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:15.344046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dae8f-425b-4d9d-807e-6f836728586c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.396758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.398409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.398936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.399392 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.400252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:15.400956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.403582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.419581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 291/579] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:15.421048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.432111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 289/580] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:15.432983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.475503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:15.575209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af55506-11b4-4527-91b6-7bae1a3d36c0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.581885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:15.626704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74face0f-5ee1-40c7-af45-7189d0b7d904 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.643368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.643991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.644266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.644536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.645177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:15.645474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.646856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.658972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 292/581] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:15.663261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.696758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.697374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.697655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.698209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.698952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=b45a976b6672412f92f9a4f8fbc75b1b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:15.699282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.700724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.716288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 290/582] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/role_assignments?user.id=b45a976b6672412f92f9a4f8fbc75b1b&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:15.716927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.827029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2f472-65dc-4fe9-be26-cf3a50a86b37 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.834111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:15.894824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.895559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.895834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.896134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.896872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] RBAC: Adding query filter params (interface=public, service_id=b2ca112e6a284dc7b2c39aeb3623f249, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:15.897176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.898602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.917534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 293/583] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/endpoints?service_id=b2ca112e6a284dc7b2c39aeb3623f249&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:15.922137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.925627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313433-38fa-4db5-af50-671c1f56c66a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:15.964829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:15.965477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:15.965754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:15.966020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] PATH_INFO: `/v3/services/network` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:15.967543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:15.970089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:15.979213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:15.980290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:15.981193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:15.984966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 291/584] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:15.987390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:15.990226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:16.054201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34ddc-d136-46fb-971d-adf74a7aae01 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:16.100675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:16.101297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.101574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.102095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.102724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:16.104829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:16.108375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679b72-a7be-42e1-89d3-7208e3ec8160 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:16.115215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 294/585] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:16.150625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:16.151367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.151646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.151917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.152577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] RBAC: Adding query filter params (name=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:16.152879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:16.154294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:16.165093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 292/586] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:16.166772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:16.207417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:16.319532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053d5e-e7c6-47f6-875d-ca8d44f5292d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:16.356314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:16.357287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.357561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.357828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.358507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] RBAC: Adding query filter params (type=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:16.358806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:16.360256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421e4a-2495-4fc8-a652-b5b99c1f4978 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:16.369295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 295/587] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:16.372933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:16.409273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b930-592f-47ce-8de1-a76863ec866e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.409591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b930-592f-47ce-8de1-a76863ec866e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.409816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7b930-592f-47ce-8de1-a76863ec866e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.411192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 296/588] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:16.417391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.417672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.417940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.439631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:16.440833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:16.463251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:16.516211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a7e8-d73f-4022-8288-c3c32e6d5d27 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:16.528365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 297/589] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:16.561201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:16.561829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.562109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.562379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.563085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] RBAC: Adding query filter params (interface=public, service_id=d15c2909811f472bbcc5e6d7da0b474b, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:16.563418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:16.564806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:16.577836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 293/590] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:16 2022] GET /identity/v3/endpoints?service_id=d15c2909811f472bbcc5e6d7da0b474b&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:16.579270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:16.607556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:16.707536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa023fa5-2051-43b6-8af5-6386f034d120 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:16.755897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:16.756892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:16.757410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:16.757869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:16.758680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:16.760524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:16.772020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 298/591] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:16 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:16.867602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.010878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf113a-59f0-4b99-8cb1-075533023215 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:17.047215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:17.053390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:17.054096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.054782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.055564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.068225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:17.073616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:17.074545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:17.075406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 294/592] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:17.080976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.182077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:17.192337 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf231b4c-10bb-4ecb-a447-f763bed17e8e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:17.229124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:17.229732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.230013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.230287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.230964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:17.231300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:17.232682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:17.241687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 299/593] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:17.247678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.398037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd57fa-787f-42cd-b53f-794697d64d0f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:17.439483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:17.440125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.440463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.440732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.455275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:17.456760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:17.480396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:17.481410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:17.482308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:17.482308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:17.482308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:17.482308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 295/594] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:17.487418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.633446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca417425-ad43-4787-bd74-5925ab7fc44f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:17.679289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:17.679844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.680116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.680386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.681039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:17.681341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:17.682703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c7087-dba4-46a8-93fe-ea34c67d005e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:17.695537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 300/595] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:17.701168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.745028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:17.779126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362ac36-a5ec-44b3-90fe-424451576cbc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.779448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362ac36-a5ec-44b3-90fe-424451576cbc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.779681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362ac36-a5ec-44b3-90fe-424451576cbc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.781831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 301/596] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:17.787445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.787724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.787991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.808760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:17.810201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:17.819847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:17.822086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf8081-60e2-493b-a334-f5d642beaacb None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:17.868606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:17.869251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:17.869529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:17.869847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:17.878170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:17.879737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:17.891331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:17.892418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:17.893342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 296/597] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:17.900168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:17.957786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 302/598] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:18.014343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d257347-87a2-4fe5-8c4d-56c61c0fb21a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:18.015899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.060045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:18.060665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.060942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.061254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.061988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:18.062293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:18.066300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:18.102736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 297/599] 10.222.0.36 () {56 vars in 1135 bytes} [Tue Oct 25 13:12:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:18.108221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.272183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc0525-9b34-4896-8271-bb4f1510cb3c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:18.320779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:18.321400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.321680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.321950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.331499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.336643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.395848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:18.397355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:18.542089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294423c-7612-46aa-8451-9f71da58a071 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:18.579087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 303/600] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:12:18.585710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.611817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:18.612469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.612748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.613015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.635319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-c49ea824-b40b-4e2e-8498-5a10717017d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:18.640499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:18.641447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:12:18.642309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:18.643885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.650684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 298/601] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:18.650684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca0bbb-d833-4832-8962-499537529413 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.650684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca0bbb-d833-4832-8962-499537529413 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.650684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca0bbb-d833-4832-8962-499537529413 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.653319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 299/602] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:18.654375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca0bbb-d833-4832-8962-499537529413 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.685347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.715082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5748fd-0de4-4bce-afc1-bcaf8e809389 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:18.752065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:18.752900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.753683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.754101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.767959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:18.769617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:18.770216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca0bbb-d833-4832-8962-499537529413 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-06b7709c-3cd4-4c3d-9f82-fcbaeee39a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:18.789293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:18.790386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:18.791349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:18.791349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:18.791349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:18.792501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 304/603] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:18.793620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.793895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.794160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.812606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:18.813255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:18.813748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:18.814152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:18.815009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:18.815480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:18.817038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:18.817845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:18.819441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:18.831273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 300/604] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:18.832549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.889628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 305/605] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:18.891078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:18.916827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.934478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:18.958813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3903459-2b1f-4896-b6a7-faf2c6f17d94 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.001971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.002612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.002895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.003189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.003731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37df3ca-c280-4f3f-b6cc-88640f075736 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.003905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.004173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.005625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.021932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 301/606] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.028257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.047507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.048555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.049181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.049796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.062101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.064155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:19.082179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:19.083221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:19.084117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:19.084117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:19.084117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:19.086219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 306/607] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:19.092233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.137161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926c2b1-19db-4bed-91df-df4876e827e8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.173274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.173915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.174614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.174902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.195789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.197255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2dc666b0-9ea3-4202-851b-b85b8aa68d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:19.212522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:19.213467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:19.214353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:19.216825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 302/608] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:19.219616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a263b6-2096-4768-a54c-7024a2932762 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.220121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a263b6-2096-4768-a54c-7024a2932762 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.220606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a263b6-2096-4768-a54c-7024a2932762 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.226653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 303/609] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:19.226964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a263b6-2096-4768-a54c-7024a2932762 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.229221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.231022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.260634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002251b-2e2d-47de-9cc1-89f5ce855ac4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.300855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.301693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.301970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.302279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.303089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.304189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.305638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775662c0-3e3c-4721-97f8-f47bc3490266 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.320435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 307/610] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.321453 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.321733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.322001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.338672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a263b6-2096-4768-a54c-7024a2932762 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.341860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:19.343233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:19.377564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.378183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.378455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.378720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.379734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.379734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.381239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.399592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 304/611] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.401067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.423236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 308/612] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:19.423236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.478546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.514023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.524219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad3039-1fe0-4b6f-b7bd-871cb4fc7103 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.549143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f937-97ed-4791-a212-e5c4f2253678 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.562354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.563093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.563751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.564039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.564919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.566317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:19.576424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:19.577515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:19.578415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 305/613] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:19.579237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.593425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.594060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.594340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.594696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.602096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.603870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-24c745c8-b454-48ac-b91c-c2de740bd1ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:19.616422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:19.617481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:19.618625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:19.620660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 309/614] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:19.625322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a31d4-00ac-4fdd-baaa-25ab1a6750ee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.625322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a31d4-00ac-4fdd-baaa-25ab1a6750ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.625322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a31d4-00ac-4fdd-baaa-25ab1a6750ee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.628027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 310/615] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:19.628989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a31d4-00ac-4fdd-baaa-25ab1a6750ee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.699346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e904d03-bf77-42e6-ad58-14734e805683 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.724706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.732464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a31d4-00ac-4fdd-baaa-25ab1a6750ee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.734649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.735271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.735551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.735819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.736544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=084038e0639b49c99485f53463392d11) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.736851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.738287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.744058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 306/616] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/role_assignments?user.id=084038e0639b49c99485f53463392d11&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.744822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.772631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:19.786204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.786944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.787341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.787675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.788383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.788745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.790209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81c169-1545-4441-aa4b-f651e2aaa9de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.800173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 311/617] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.800173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.800173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.800173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.817459 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:19.818594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:19.862446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb829b43-9736-4707-9b7a-5bbac1cf3b6a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:19.891189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 312/618] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:19.891990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:19.913508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:19.914803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:19.915262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:19.915769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:19.916661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:19.917137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:19.919193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:19.940587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 307/619] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:19.941427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.001149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.019085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9029faf-409f-4679-aa05-441a9a1d48b1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.045389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.053651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfc562-dbf6-49f4-a7cd-bed76c0151a9 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.068705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.069663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.070176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.070730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.071538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.073345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:20.084049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:20.085105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 25 13:12:20.085968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:20.088452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 313/620] 10.222.0.36 () {56 vars in 1129 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:20.089381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.096807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.097682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.098102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.098546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.099386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:20.099829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.101402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.107195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 308/621] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:20.108197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.250404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017e10-d64f-41d1-ba59-61bae704dfad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.251880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a7f40-5f96-497a-b892-962fbed459d5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.266112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.290171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.290715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.290831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.291060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.291378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.291846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.292263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.292663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.293430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:20.293845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.295412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.301419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 309/622] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:20.302409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.359944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.381526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.383328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.423479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1803f3e-5cb7-4ec6-9646-7988761a9c40 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.461903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.462542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.462820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.463088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.463714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.465142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:20.474279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:20.475331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 25 13:12:20.476171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:20.480347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 310/623] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:20.481234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.520820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 314/624] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:12:20.521829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.536870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.614318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff985d9-0dba-407a-9a7d-5cd280ccf7b6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.620163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.640871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97111626-f3be-46df-a642-5aa50a303d8e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.654737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.655420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.655701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.655971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.656617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:20.656917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.658481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.664914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 311/625] 10.222.0.36 () {56 vars in 1124 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:20.665725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.676767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.677384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.677706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.677979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.678619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:20.678917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.680390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.689938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 315/626] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:20.690882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.781818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f5106-6c8c-410c-b079-892fa86778c6 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.785058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:20.804156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58129775-2133-4bef-a5d6-bcc0f274edc7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:20.820825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.821805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.822143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.822936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.835602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.837314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.846696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:20.847823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:20.848117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:20.848393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:20.849367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:20.850933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:20.854329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:20.856300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:20.858224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:20.858224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:20.858224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:20.858224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 312/627] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:20.858224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.880034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 316/628] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:20.880996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:20.886535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:21.085639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eed4e-e620-4fc7-b0a9-b039369949b3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.093191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b945f-326f-4011-84d1-b346193e3769 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.123093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.124293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.124751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.125196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.126173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:21.126687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.128345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.147268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 317/629] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:21.147437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.147437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.147437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.147437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.148030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] RBAC: Adding query filter params (type=network) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:21.148331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.149714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.153092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:21.162562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 313/630] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:21.167872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:21.193968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:21.237634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:21.285966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc49b5-ff65-47a0-a44a-8ff9facce30e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.328397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bf071-2cb8-4aa8-a8d4-d25737a53319 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.331888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.332529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.332993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.333118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.333863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.335316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.384204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 314/631] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:21.404051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.404640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.404957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.405229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.417856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.419455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:21.432466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:21.433428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:21.434347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 318/632] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:21.440016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:21.561562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:21.594567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eee92c-fb0a-4dd4-875d-a044d542ef7f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.639494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.640118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.640397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.640674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.641383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:21.641685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.643177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77175e-faeb-4b87-b21d-7fc23fc8ca38 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.656146 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 315/633] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:21.661884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:21.785736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d8d9-f91c-41da-8734-e13dc5a79933 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:21.804602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:21.825130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:21.825798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:21.826064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:21.826358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:21.827235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=5afba0fee3f74a5883257487f6c5c84b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:21.827541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:21.829051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2554f1-bde2-4fdf-80ec-d6fdccc9dca6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:21.835996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 319/634] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:21 2022] GET /identity/v3/role_assignments?user.id=5afba0fee3f74a5883257487f6c5c84b&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:22.200110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f3cf5-f561-4100-b672-397cdf4a883b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.200644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f3cf5-f561-4100-b672-397cdf4a883b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.200784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f3cf5-f561-4100-b672-397cdf4a883b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.202583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 316/635] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:22.208383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca16-1329-469b-8f63-85b944f34919 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.208770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca16-1329-469b-8f63-85b944f34919 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.209155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ca16-1329-469b-8f63-85b944f34919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.227625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9d27ca16-1329-469b-8f63-85b944f34919 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:22.228901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9d27ca16-1329-469b-8f63-85b944f34919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:22.294381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 320/636] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:22.466450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44455-e0b2-4011-ab38-186547d3952e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.467073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44455-e0b2-4011-ab38-186547d3952e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.467664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44455-e0b2-4011-ab38-186547d3952e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.469482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 317/637] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:22.474896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.475774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.476244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.494978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:22.496508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:22.565212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a44455-e0b2-4011-ab38-186547d3952e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:22.569511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 321/638] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:22.673450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a44455-e0b2-4011-ab38-186547d3952e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:22.707708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:22.708352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.708625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.708948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.709538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:22.710923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:22.713525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:22.720876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:22.721945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:22.723024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 318/639] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:22.728367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:22.771056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:22.832237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f3f7a-9def-48b4-b3f1-f75b2dc63402 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:22.865924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:22.866530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:22.866805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:22.867130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:22.867803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:22.868148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:22.869529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:22.874427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 322/640] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:22.878860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:22.880359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:22.989664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dd97-93d2-4a6f-8a06-3f7c57dbbde2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:22.993336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:23.002458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d6e08-6a37-44c8-a094-2fe7f7bc2b4a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:23.037328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:23.038099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:23.038473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:23.038837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:23.039610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:23.040025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:23.041561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750b48b-3f88-4dab-8414-d4ba842a2db1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:23.044846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:23.045677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:23.046096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:23.046505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:23.047325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:23.048908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb7dbc-d17a-4514-8626-23ff9385de26 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:23.055481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 323/641] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:23.073807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 319/642] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:22 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:23.435670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb71c2-0741-4501-8ef0-b619cdc70ff8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:23.436090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb71c2-0741-4501-8ef0-b619cdc70ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:23.436302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb71c2-0741-4501-8ef0-b619cdc70ff8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:23.437751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 324/643] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:23.443290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:23.444152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:23.444646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:23.464344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:23.465888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:23.530555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:23.551830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 320/644] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:23.828584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb71c2-0741-4501-8ef0-b619cdc70ff8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:23.937785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb71c2-0741-4501-8ef0-b619cdc70ff8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:23.973889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:23.974920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:23.975426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:23.975951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:23.976780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:23.978410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:23.987774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:23.988816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:23.989676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:23.992571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 325/645] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:23.998789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:24.106659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7f1ad-1a6f-4047-8efb-aebc45052000 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:24.144442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:24.145450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.145956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.146501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.147367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:24.147869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:24.149440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ece4e-d3e4-4ceb-b4c6-7669c78a33fc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:24.154698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 321/646] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:24.160675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:24.193363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:24.248889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e45f8-f363-4ad3-9c8e-dfbf6a5f131e None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.249240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e45f8-f363-4ad3-9c8e-dfbf6a5f131e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.249440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e45f8-f363-4ad3-9c8e-dfbf6a5f131e None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.251098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 322/647] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:24.256273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.256558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.256938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.270445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e1c1c-2a2e-41bc-bcc6-3b8ab98d67c1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:24.275778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:24.276972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:24.312363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:24.313492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.314068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.314544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.315608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:24.316106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:24.318548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:24.324188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 326/648] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:24.339062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:24.366468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 323/649] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:24 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:24.651722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:24.748241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:24.755297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd638c-04ba-4dc2-8cda-fcac58f9b78c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:24.789712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:24.790343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.790623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.790894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.801371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:24.806654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:24.807605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:24.808826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 327/650] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:12:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:24.814143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:24.917061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822fb33-f8bd-4738-bff7-eda404041f87 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:24.953400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:24.954060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:24.954357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:24.955001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:24.955716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:24.956019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:24.957398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:24.966053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 324/651] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Oct 25 13:12:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:24.971805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.076921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa073c-dec4-427a-b1bf-410b0df5ccf1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.112768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.113407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.113693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.113971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.124939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.126644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:25.141725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:25.142679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:25.144017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:25.144017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:25.144017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:25.144017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 328/652] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:25.148907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.194967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:25.262879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084ba1-5d8d-455f-a53c-3569b6a4b988 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.302281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.303336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.303930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.304389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.305245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:25.305735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.307365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5366f-c487-462b-861e-ad1dfd34301a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.320932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 325/653] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:25.326209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.339703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:25.346258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc39f-8c93-41e0-bc7f-ab0ac4cb5e24 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.346866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc39f-8c93-41e0-bc7f-ab0ac4cb5e24 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.347873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc39f-8c93-41e0-bc7f-ab0ac4cb5e24 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.350047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 326/654] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:25.354969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac296cb7-4447-416a-b438-6c718f930537 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.355271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac296cb7-4447-416a-b438-6c718f930537 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.355573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac296cb7-4447-416a-b438-6c718f930537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.377958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ac296cb7-4447-416a-b438-6c718f930537 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:25.379708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac296cb7-4447-416a-b438-6c718f930537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:25.437323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5b9f-a51c-434d-9c32-5dca487012f8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.455184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 327/655] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:25 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:25.456468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e22d-3d49-4688-90b1-3b456f0856a8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.456714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e22d-3d49-4688-90b1-3b456f0856a8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.457010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e22d-3d49-4688-90b1-3b456f0856a8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.459180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 328/656] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:25.463732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.464017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.464289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.485544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:25.486065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.486736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:25.487251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.487720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.488231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.498312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.500240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:25.512733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:25.513724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:25.514691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:25.516410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:25.518079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 329/657] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:25.523540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.567311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 329/658] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:25 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:25.607774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:25.645982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce25bc-05be-4117-a064-700905589bc7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.680322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.681180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.681600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.682007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.682879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:25.683376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.684958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.706566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 330/659] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:25.711887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.734917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.823807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85c29f-1005-44b5-a333-fe54e859579a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.851543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d09a-0d40-4bc3-821d-4e6b2ab6b7e1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:25.859955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.860582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.860856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.861118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.872013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.873492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.880511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:25.888050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:25.889016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:25.889962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:25.889962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:25.889962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:25.889962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 330/660] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:25.893057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:25.893979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:25.894455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.894655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:25.895110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:25.895906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:25.897304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:25.907137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:25.908082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:25.909003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 331/661] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:25.909798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:25.915625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.007878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e89121-85cb-42ab-b9c5-51784484777b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.012276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0316e3b-afdb-4d6f-a6e6-b3747f72a960 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.045252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.045471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.045891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.046058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.046294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.046391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.046549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.046657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.047236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.047329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:26.047600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.048620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.048980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.061822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 332/662] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:26.062660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.075333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 331/663] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:25 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:26.075333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.141160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.167857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.207163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc992e5-45b3-49ce-88f1-d7a8f60ae67a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.218701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd6c26-a676-4fd2-b3d6-129e98a11cf5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.254646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.255423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.255758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.256092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.259008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.259606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.259916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.260200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.261837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:26.261837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.262540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.264016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.266235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.273868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 333/664] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:26.274798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:26.278288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:26.279218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:26.280186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 332/665] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:26.285817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.393039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0712a-804e-4195-a9a3-dfb539f138f8 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.411994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb95db-90eb-4bfc-aca5-65ee06db9f17 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.431772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.444617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.446105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.447487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.448098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.448450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.449176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:26.449812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.451534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.453985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.455283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.455802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.456258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.457148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:26.457749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.457854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 334/666] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:26.459554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.463562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.474296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 333/667] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:26.480649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.590252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634db5f0-a6c5-45b6-931a-96d394d5b841 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.604016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06e7eb-a7e7-4810-936f-f9cd65bf5111 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.631816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.632447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.632727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.633002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.633714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] RBAC: Adding query filter params (interface=public, service_id=edd0875defaa4d89aed7aabab3c0f987, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:26.634026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.635468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c8ca9-1c6d-4dd2-bc9f-0d83905f1448 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.641005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 335/668] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/endpoints?service_id=edd0875defaa4d89aed7aabab3c0f987&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:26.644553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec72-ff33-47a4-9777-81b8e7e51ed8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.645042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec72-ff33-47a4-9777-81b8e7e51ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.645574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bec72-ff33-47a4-9777-81b8e7e51ed8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.646942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.647401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 336/669] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:26.647694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.647973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.648241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/5afba0fee3f74a5883257487f6c5c84b/roles/840f7fde7a13466c8804a7a41a340461` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.648924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bec72-ff33-47a4-9777-81b8e7e51ed8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:26.652997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=5afba0fee3f74a5883257487f6c5c84b, role_id=840f7fde7a13466c8804a7a41a340461)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.655391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce1a59-cc53-4a64-a37c-4be50b827f50 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.679177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 334/670] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:26 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/5afba0fee3f74a5883257487f6c5c84b/roles/840f7fde7a13466c8804a7a41a340461 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:26.679177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.679177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.679177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.703349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:26.705823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:26.731571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.765462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bec72-ff33-47a4-9777-81b8e7e51ed8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:26.772490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:26.803099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 335/671] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:26 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:26.806564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:26.811210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:26.811210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:26.811210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:26.811210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:26.811210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b4d6b-7ea1-47c7-b8c2-5e0b5f0479e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:26.817994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 337/672] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:26 2022] GET /identity/v3/services => generated 1258 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:12:27.077217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:27.177748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28bce3-09c9-41e8-bf61-38a67b75c003 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:27.212637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:27.213252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:27.213528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:27.213791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:27.214462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:27.215849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37d4ec-12f3-4ed3-8eca-23aeca69e5d5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:27.232021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 336/673] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:27 2022] POST /identity/v3/services => generated 251 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:28.287913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f32a25-60c8-4f6c-8875-ba76c38cb473 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.288475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f32a25-60c8-4f6c-8875-ba76c38cb473 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.288475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f32a25-60c8-4f6c-8875-ba76c38cb473 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.290052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 338/674] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:28.295342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.296151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.296665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.316033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:28.317511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:28.352657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:28.393294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 337/675] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:28.659134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f32a25-60c8-4f6c-8875-ba76c38cb473 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:28.765773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f32a25-60c8-4f6c-8875-ba76c38cb473 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:28.774493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:28.803960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:28.804660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.804938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.805205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.805794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:28.807182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3e02176f-76cc-4336-8e6a-96b3e5436290 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:28.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:28.817241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:28.818121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 339/676] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:28.824505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:28.889002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5fe8-b755-4373-b775-e64991d1d980 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.889544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5fe8-b755-4373-b775-e64991d1d980 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.889819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d5fe8-b755-4373-b775-e64991d1d980 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.891539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 340/677] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:28.898165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcbba7-d914-4ef2-a9ba-53c22239003e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.898469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcbba7-d914-4ef2-a9ba-53c22239003e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.898705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcbba7-d914-4ef2-a9ba-53c22239003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.928076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a8fcbba7-d914-4ef2-a9ba-53c22239003e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:28.929566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8fcbba7-d914-4ef2-a9ba-53c22239003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:28.932856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c747ac-ebbf-4562-b2e7-83d8333f9a0c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:28.969097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:28.969710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:28.969989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:28.970247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:28.970882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:28.971258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:28.972682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:28.982713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 338/678] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:28.983738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.010853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 341/679] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Oct 25 13:12:29.022823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.030318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0abed-e9a2-4533-bc9a-e32ab617983e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.030907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0abed-e9a2-4533-bc9a-e32ab617983e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.031507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0abed-e9a2-4533-bc9a-e32ab617983e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.033312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 342/680] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:29.038593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.038866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.039130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.060790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:29.062157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:29.105708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0d09b-f85d-4511-978f-2df1cb154e20 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.139249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 343/681] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:29.148029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.148639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.148913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.149179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.149810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:29.150106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.151635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.157043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 339/682] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:29.162671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.198086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.267546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2c047-9388-47b0-99cc-f7813cdaaafa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.279424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.303054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.303746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.303967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.304226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.304925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] RBAC: Adding query filter params (interface=public, service_id=1b1523efe66141119e7c4b1fb96c2cee, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:29.305223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.306598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.312112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 344/683] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/endpoints?service_id=1b1523efe66141119e7c4b1fb96c2cee&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:29.317490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.393265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.403653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df076a-837a-4f66-a97a-e21a31b56c1d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.427385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.435428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a5633-8021-4b53-a252-9ea311559b56 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.446223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.446871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.447167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.447451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.458185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:29.463055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:29.469154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 340/684] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:29.470222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.480948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.481633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.481893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.482389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.482844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.484292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.499540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 345/685] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:12:29.500396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.583106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90dc7e-8e2a-4729-8e2e-6836941c9034 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.618265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f760b-2e66-47db-ac70-2cfd31691037 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.635653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.636605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.636881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.637152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.637759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.639262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-890af45f-e7ac-4bc2-92b8-b7a781586dc4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:29.649035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:29.650023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:29.650919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 341/686] 10.222.0.36 () {56 vars in 1132 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:29.652773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6906e-dc5a-47f8-bcbf-5ff0282db87a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.652773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6906e-dc5a-47f8-bcbf-5ff0282db87a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.652773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6906e-dc5a-47f8-bcbf-5ff0282db87a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.657669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 342/687] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:29.657669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6906e-dc5a-47f8-bcbf-5ff0282db87a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.667117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.667770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.668045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.668313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.669000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:29.669300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.670778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de23a7-77ba-4331-8ba4-0f08ba2dc567 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.678444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.680552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 346/688] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:29.681370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.682006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.682006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.701236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:29.702483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:29.717906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.771730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6906e-dc5a-47f8-bcbf-5ff0282db87a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.772094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 347/689] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:29.772914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.813168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.814153 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.814619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.815099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.816069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:29.816627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.818058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.827185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 343/690] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:29.830741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:29.913821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217e96d-0a82-4b84-a033-5e71ba10cf57 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.917184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:29.937536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597d22-3376-4936-992c-1fe032c43f26 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:29.965044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.966298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.967613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.972825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:29.973811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:29.974083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:29.974347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:29.974980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:29.975302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.976691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.981377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:29.982635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 344/691] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:29.983393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:29.988823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:30.003211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:30.004287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:30.005182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:30.005182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:30.005182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:30.008234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 348/692] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:30.018574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.019041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.099126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ead2a3-1235-4279-87b6-83d6cb18ca52 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.129633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf35b4a-0baf-46d1-99ce-0f7bfdc51289 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.142817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.143553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.143832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.144097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.144780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.146147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.164821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.165367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.165639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.165937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.166654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:30.166988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.168469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.171684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 345/693] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:30.172749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.181609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.183010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 349/694] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:30.189823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.287982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.307811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf53f7-9807-433f-b7c1-7607c9a5f364 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.346335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e1b7-9cf0-439b-9559-c0558ea53220 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.349920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.350758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.351198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.351607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.352377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.354050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:30.364342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:30.365520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:30.366377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:30.374101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 346/695] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:30.375370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.389662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.390426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.390939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.391374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.401161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.402673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:30.418138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:30.419129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:30.420125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 350/696] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:30.425004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.457993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.507840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475528d2-4393-4ffe-a264-a3171195852f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.536356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1f9ff-b132-4958-bc49-5f4bf6e8e31e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.551298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.551852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.552127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.552395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.553099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:30.553423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.555643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.561502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 347/697] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:30.566522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.568088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.582416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.583674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.584351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.584771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.585942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:30.586414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.588721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.613000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 351/698] 10.222.0.36 () {56 vars in 1149 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:30.619973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.698189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197aa24-c567-4733-9613-f22ec3e5d13e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.726229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65ec9-26d6-4c36-88bd-bc577a774712 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.729479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.741625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.742250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.742524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.742786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.743812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:30.744113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.745483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.752006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 348/699] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:30.758024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.773714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.774676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.775228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.775689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.825425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.855011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.857205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.877038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116c6e9-d95f-4c74-a5d9-631cbedfcd65 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:30.915996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:30.917365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:30.918142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:30.918609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:30.919602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] RBAC: Adding query filter params (interface=public, service_id=1de630d9f69a4083ac5f1889035c740e, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:30.920105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:30.921795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:30.928447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 349/700] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/endpoints?service_id=1de630d9f69a4083ac5f1889035c740e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:30.935296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:30.990052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 352/701] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 25 13:12:30.994057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:30.995774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:31.063898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f0013-db57-4558-a442-0fe68ddfb408 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.083301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:31.104594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33a5b7-d0c9-4240-856b-5825c89a8e5c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.113287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.114638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.115258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.116138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.117233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.119389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.135580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 350/702] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:12:31.146634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.147719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.148204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.148669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.161876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.163909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:31.181622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:31.182674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:31.183698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:31.183698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:31.183698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:31.186308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 353/703] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:31.191861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:31.307797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c985ad-7c52-4a76-b4fa-983ef576706e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.327253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:31.350795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.350795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.350795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.350795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.351423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:31.351423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.352514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.364918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 351/704] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:31.370323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:31.477700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bdae9-0f36-45df-a262-f520ff9082b0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.511952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.512601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.512887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.513156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.520181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.520680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:31.522304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e9c32787-01a9-4732-8ba7-edddd2210c6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:31.534580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:31.535574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:31.536482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:31.544700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 354/705] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:31.550985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:31.664059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d697-d254-4da1-b106-8f5ec17fd0a7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.698949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e816b8-c221-443a-8523-806f0ccbfe51 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.699288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e816b8-c221-443a-8523-806f0ccbfe51 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.699526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e816b8-c221-443a-8523-806f0ccbfe51 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.701025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 355/706] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:31.705742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.706606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.706781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.707044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.707341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.707687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.708135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.709019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:31.709522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.711278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.724451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 352/707] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:31.730283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:31.731588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:31.732802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:31.776974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:31.823172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 356/708] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:31.843870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804b2b2-0591-43a7-bd95-9ee951ab3827 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:31.878827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:31.879397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:31.879674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:31.879941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:31.880672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] RBAC: Adding query filter params (role.id=840f7fde7a13466c8804a7a41a340461, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=5afba0fee3f74a5883257487f6c5c84b) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:31.881044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:31.882470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:31.888914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 353/709] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:31 2022] GET /identity/v3/role_assignments?user.id=5afba0fee3f74a5883257487f6c5c84b&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=840f7fde7a13466c8804a7a41a340461 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:32.093459 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:32.197780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26b7ae-e486-4dc0-8075-255e6e61db2b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:32.222398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:32.235115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:32.236077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.236554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.237057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.237837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:32.239439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:32.248991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:32.250024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:32.250868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:32.253828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 357/710] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:32.260034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:32.364397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20c80-7010-42ba-b10e-bde6dd0add57 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:32.366274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:32.401986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:32.402604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.402871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.403158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.403868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:32.404174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:32.405959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4141e-ab29-4e82-b991-7c960763ea2b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:32.411115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 354/711] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:32.416657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:32.527668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572275-381f-4bac-9742-193a6c90b1f1 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:32.558873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82cd0-4594-4a9a-b27a-3f29212e8866 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.559438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82cd0-4594-4a9a-b27a-3f29212e8866 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.559438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82cd0-4594-4a9a-b27a-3f29212e8866 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.561024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 355/712] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:32.563887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:32.564501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.564774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.565041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.565688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:32.565988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:32.567496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:32.567943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.568547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.569038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.573265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 358/713] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:32.578944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:32.589025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:32.590342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:32.637876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:32.671672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 356/714] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:32.704059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922b577-0bd4-4e9c-a385-5f177d8b7c4c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:32.739354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:32.740330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:32.740804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:32.741298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:32.742201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:32.743820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:32.771597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 359/715] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:32.943985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:33.039179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:33.046499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd7f7f-8606-4346-9ce1-1057e1ef9549 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:33.087110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:33.087779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.088053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.088330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.088900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:33.090260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:33.099182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:33.100156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.101151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 357/716] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:33.109017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:33.212498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151bc56-8799-4480-b84a-75fb0190c6b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:33.220256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:33.252234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:33.253142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.253568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.253981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.254763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:33.255252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:33.256827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392a30a-b1dd-4837-bc8c-c58951ccb3d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:33.262444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 360/717] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:33.268584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:33.381670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31382f-5c2c-4917-92e4-981a9db27143 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:33.392971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93e9a1-3a75-4769-80b1-482106ac6d82 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.393256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93e9a1-3a75-4769-80b1-482106ac6d82 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.393531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93e9a1-3a75-4769-80b1-482106ac6d82 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.398827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 361/718] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:33.399947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.400240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.400517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.419355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:33.420620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:33.422135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:33.423049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.423386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.424541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.425186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:33.425487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:33.426865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:33.433242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 358/719] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:33.502232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 362/720] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:33.775081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:33.800642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:33.879917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045c2aa-34f1-45d9-8ca5-9618cc19b09b None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:33.913059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:33.913693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:33.914028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:33.914298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:33.914894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:33.916311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:33.925227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:33.926311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:33.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 359/721] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:33.932753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.043494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:34.047171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e585dc-4c61-455d-93fc-33361bc8bbcc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:34.083699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:34.084644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.085112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.085570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.086398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:34.086883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:34.088467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fd7d9-1067-42c5-b62c-01a76a3efaaf None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:34.094114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 363/722] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:34.099995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.126347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86a03-8dc9-4956-bab3-6116e6ea19f8 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.126687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86a03-8dc9-4956-bab3-6116e6ea19f8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.126902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86a03-8dc9-4956-bab3-6116e6ea19f8 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.129769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 364/723] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:34.135184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.135726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.136205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.157831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:34.159034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:34.200758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:34.218702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7f182-abda-43cb-9f10-3340343003dd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:34.254383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:34.254990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.255361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.255555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 365/724] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:34.255668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.256312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:34.256618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:34.257990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:34.263309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 360/725] 10.222.0.36 () {56 vars in 1114 bytes} [Tue Oct 25 13:12:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:34.272477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.372014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:34.379797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0ffa-84bd-403f-9783-c45687f6b5d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:34.415218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:34.416076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.416515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.416927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.417746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:34.419306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:34.443612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 366/726] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:34.530017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.645719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ca12f-bc91-4db8-be3e-5c7439e2e88c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:34.691032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:34.691692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.691972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.692241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] PATH_INFO: `/v3/services/image` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.692829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:34.694202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:34.703397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:34.704401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:34.705605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 361/727] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Oct 25 13:12:34 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:34.711198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.816564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d1979-5dd7-4b16-8266-412eb5a0af30 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:34.852487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:34.853482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:34.853960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:34.854432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:34.855299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:34.855790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:34.857350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9a18-6233-4603-a9c9-f16aae553e95 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:34.863029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 367/728] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:34.875210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:34.880263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:34.971821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4f6c-3da5-48ce-9d94-da0483fad383 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:35.005652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:35.006312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.006555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.006828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.007521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] RBAC: Adding query filter params (type=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:35.007830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:35.009369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98fa60-c39e-4a9b-999a-4899dbad7012 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:35.014401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 362/729] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:35.152931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee6b0-37af-4775-af6e-2016a22fa226 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.153113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee6b0-37af-4775-af6e-2016a22fa226 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.153386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee6b0-37af-4775-af6e-2016a22fa226 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.155052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 368/730] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:35.161104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d272f-a481-4c07-8c7a-ee91c6b97559 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.161391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d272f-a481-4c07-8c7a-ee91c6b97559 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.161665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d272f-a481-4c07-8c7a-ee91c6b97559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.181404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-977d272f-a481-4c07-8c7a-ee91c6b97559 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:35.182628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-977d272f-a481-4c07-8c7a-ee91c6b97559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:35.253730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 363/731] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:35.519892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ee6b0-37af-4775-af6e-2016a22fa226 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:35.538081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:35.630808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ee6b0-37af-4775-af6e-2016a22fa226 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:35.665152 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:35.665784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.666056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.666322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.666905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:35.668319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:35.672178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fdccf-28d0-489c-b760-1e9fb42828ae None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.672440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fdccf-28d0-489c-b760-1e9fb42828ae None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.672713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fdccf-28d0-489c-b760-1e9fb42828ae None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.674337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 364/732] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:35.677654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:35.678679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:35.679575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 369/733] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:35.681022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.681309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.681570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.688180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:35.690753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:35.704993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:35.706270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:35.770560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:35.779351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 365/734] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:35.794055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbce30-c052-4084-9290-b9da367c4833 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:35.828825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:35.829442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.830083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.830349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.830986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:35.831478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:35.832870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:35.838240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 370/735] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:35.844284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:35.944116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15beb8-1f45-49e6-a420-b74ad6c6f1f7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:35.977051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:35.982000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:35.982624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:35.982925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:35.983217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:35.983861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:35.984168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:35.985555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:35.990747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 366/736] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:35.996370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:36.063795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:36.096341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6b234-39e1-46ce-a8fd-4498c6199ac2 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:36.130047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:36.130666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:36.130942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:36.131236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:36.132112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] RBAC: Adding query filter params (interface=admin, service_id=1b1523efe66141119e7c4b1fb96c2cee, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:36.132427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:36.133800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:36.139921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 371/737] 10.222.0.36 () {56 vars in 1249 bytes} [Tue Oct 25 13:12:35 2022] GET /identity/v3/endpoints?service_id=1b1523efe66141119e7c4b1fb96c2cee&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:36.145472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:36.154054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:36.177881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5d961-70e3-4155-afae-20983237283c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:36.215881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:36.217058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:36.217562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:36.218017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:36.218866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:36.220457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c288fea-d1e9-48d9-8ad7-9e0f7aaae3ee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:36.243337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 367/738] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:36 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:36.247818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30978b-96ee-452b-adb7-c461a5bf0cfc None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:36.294254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:36.294880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:36.295321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:36.295597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:36.296212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:36.297591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2d654-be04-4f99-bf97-ad4ac392016f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:36.305122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 372/739] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:36 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:12:36.794567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b881845-7e1b-490a-943f-23a92fac93b2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:36.794948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b881845-7e1b-490a-943f-23a92fac93b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:36.795137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b881845-7e1b-490a-943f-23a92fac93b2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:36.796815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 368/740] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:36.802204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2c77-81ee-4bff-8953-e49dc0607d01 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:36.802772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2c77-81ee-4bff-8953-e49dc0607d01 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:36.802772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2c77-81ee-4bff-8953-e49dc0607d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:36.821315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-35fd2c77-81ee-4bff-8953-e49dc0607d01 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:36.822491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-35fd2c77-81ee-4bff-8953-e49dc0607d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:36.885260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:36.893013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 373/741] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:37.154007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b881845-7e1b-490a-943f-23a92fac93b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:37.196479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6386ee-6111-4c89-b03d-a2a8be353724 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.196828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6386ee-6111-4c89-b03d-a2a8be353724 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.197030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6386ee-6111-4c89-b03d-a2a8be353724 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.198642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 374/742] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:37.204062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.204364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.204971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.217430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:37.228599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:37.230264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:37.268343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b881845-7e1b-490a-943f-23a92fac93b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:37.305499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:37.306104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.306383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.306651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.306830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 375/743] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:37.307393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:37.308897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:37.311772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING keystone.server.flask.common [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 25 13:12:37.330821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:37.330821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:12:37.330821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [parameters: {'id': 'f9ceae8ab22141249a4174dedcc0de49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 25 13:12:37.330821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9ceae8ab22141249a4174dedcc0de49.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9ceae8ab22141249a4174dedcc0de49. Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:37.338364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 25 13:12:37.339244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Oct 25 13:12:37.340576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 25 13:12:37.342419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 25 13:12:37.344338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:37.346499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'id': 'f9ceae8ab22141249a4174dedcc0de49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:37.352693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:37.354683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f9ceae8ab22141249a4174dedcc0de49. Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:37.356975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 369/744] 10.222.0.36 () {60 vars in 1140 bytes} [Tue Oct 25 13:12:37 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:37.358361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:37.443526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1fe6c9-d590-498a-b16d-eeca9f1217e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:37.469826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:37.479182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:37.479830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.480106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.480379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.488158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:37.489666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:37.500957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:37.501842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:37.502745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 376/745] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:37.508373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:37.603169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:37.621234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6730b8c-eaf5-45e0-810a-a48fd8647fee None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:37.636033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:37.660213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:37.660706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.660983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.661256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.662007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:37.662234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:37.664069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea9a68-6af4-446c-81a6-030aa6e92274 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:37.686345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 370/746] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:37.728065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63cb06-54db-48c1-b433-4281a69795fa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:37.764913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:37.765613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:37.765899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:37.766170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:37.766866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:37.768320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152ea71-a1f0-4648-8e13-a5c178345e14 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:37.790036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 377/747] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:37 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:38.520944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7cf36-5934-4be0-bbb5-0f332717fbd4 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:38.521480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7cf36-5934-4be0-bbb5-0f332717fbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:38.521480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7cf36-5934-4be0-bbb5-0f332717fbd4 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:38.523109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 371/748] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:38.528561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:38.528990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:38.529166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:38.542724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7e8b-0221-4e97-9275-a21ed1d4a2b8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:38.543010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7e8b-0221-4e97-9275-a21ed1d4a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:38.543322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7e8b-0221-4e97-9275-a21ed1d4a2b8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:38.544882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 372/749] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:38.549293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:38.550097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:38.550383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:38.550655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:38.551369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:38.573311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:38.574810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:38.610132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:38.647216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 378/750] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:38.649726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 373/751] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:38.910272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:38.923205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.029420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bcfdd-7580-466f-a524-ae61588b76d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.034983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d1510-d564-4fab-ac19-caa14b01ff3b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.036794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:39.074534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.074759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.075212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.075495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.075603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.075759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.076058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.076343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.076516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.077215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.077822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.078763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:39.088204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:39.089490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 25 13:12:39.091241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:39.092853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:39.094926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:39.096638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:39.108329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 379/753] 10.222.0.36 () {56 vars in 1126 bytes} [Tue Oct 25 13:12:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:39.108329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.108489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 374/753] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:39.108489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.143568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:39.205306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825ae8c-1ceb-4491-b46d-d66af3fa7247 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.231075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2224403-8e12-4b49-80a4-d5cfdf5fe419 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.242678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.243594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.243971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.244338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.245068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:39.245463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.246932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.256991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 375/754] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:39.257934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.271062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.272091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.272374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.272638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.273533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:39.273830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.275911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.283239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 380/755] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:39.289170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.297545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:39.386347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3fd21-b3b3-4f9c-8ad0-8e79134154ab None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.390221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0908246-7436-4389-b7c4-291065366a9a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.423395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:39.423981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.424821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.425285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.425731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.426617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:39.427075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.428740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.432730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.433419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.433765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.434087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.434805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:39.435179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.435914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 381/756] 10.222.0.36 () {56 vars in 1122 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:39.440113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.441658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.451307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 376/757] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:39.452783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.557156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:39.565627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5524a-0596-4335-ab26-348ec2168006 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.574238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f2008-4449-43d0-bad1-ebe91edcf9e6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.602321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.603279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.603730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.604198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.605101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] RBAC: Adding query filter params (interface=public, service_id=59814e9270e44c87a5c97a962ac3f2e8, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:39.605623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.607200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.617809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 377/758] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/endpoints?service_id=59814e9270e44c87a5c97a962ac3f2e8&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:39.618550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.619191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.619473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.619740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.619952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:39.620412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.621890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03ea3b-6d17-43ba-bd11-1c6f2aaba6fd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.659720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 382/759] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:39.749221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12825-0453-47a4-b166-9228f126dcdc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:39.794945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:39.796180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.796671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.797128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.797935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:39.799537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163c283-8017-4c18-ad11-5d565c36b153 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:39.807836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 378/760] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:39 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:12:39.981098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022be9-77f8-43e8-a052-22b67fcf4f50 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.981439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022be9-77f8-43e8-a052-22b67fcf4f50 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.981649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022be9-77f8-43e8-a052-22b67fcf4f50 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:39.983512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 383/761] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:39.989282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:39.989609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:39.989836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.009436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:40.010619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:40.024308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4b6c-6b6c-47bb-b065-1c01d41eb7bb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.024578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4b6c-6b6c-47bb-b065-1c01d41eb7bb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.024849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4b6c-6b6c-47bb-b065-1c01d41eb7bb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.025316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.026630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 384/762] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:40.031854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.032114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.032395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.053133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:40.054512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:40.075644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.092543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 379/763] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:39 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:40.123897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 385/764] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:40.362088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.393111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.483039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcf7a3-f7ce-4204-9fae-2fcb4ed7a3e8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.507196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecb57e-a9bb-44b2-a205-11382bc73247 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.530022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.531323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.531805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.532639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.533763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.536058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.547038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.548628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.548991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.549295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.549925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.551360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.552342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.sql.core [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 25 13:12:40.552342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 25 13:12:40.552342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [parameters: {'user_id': '73653b66513544d8bbe9086089b58adb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 25 13:12:40.552342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81436) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:40.560328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:40.561700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application yield resource Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.rollback() Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.force_reraise() Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise self.value Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application session.commit() Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 25 13:12:40.562684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._prepare_impl() Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.session.flush() Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._flush(objects) Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application compat.raise_( Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application flush_context.execute() Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rec.execute(self) Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 25 13:12:40.563604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:40.565752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:40.567806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [parameters: {'user_id': '73653b66513544d8bbe9086089b58adb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:40.570041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 25 13:12:40.571843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 25 13:12:40.574279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:40.575711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:40.584080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:40.586513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 386/765] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:40.589334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.589449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 380/766] 10.222.0.36 () {60 vars in 1131 bytes} [Tue Oct 25 13:12:40 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:40.589449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.589657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.603685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.684312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339514-d302-4ce3-9823-8a69b3e3025a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.702471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80da59-782c-4b59-b457-96dda83f0e67 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.722897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.723287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.723603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.723910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.724582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] RBAC: Adding query filter params (name=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:40.724931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.726465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.735305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 387/767] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:40.738298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.740362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.741092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.741429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.741883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.751031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.752898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:40.765763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:40.767005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 25 13:12:40.768396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:40.773838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 381/768] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:40.774723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.833164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.844197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:40.855025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d606fb-5ef6-40d0-9ad3-8e9741a78786 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.875826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6eb11e-9ec8-4ad8-bce4-cd308569651d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:40.892628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.893511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.893982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.894394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.895239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:40.895703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.897309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.907845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 388/769] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:40.910951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:40.911310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:40.911746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:40.912087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:40.912417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:40.913185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:40.913548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:40.915066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:40.953654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 382/770] 10.222.0.36 () {56 vars in 1101 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:41.024790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a0876-5f37-41ff-ad5f-c3b144b20cc5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:41.063918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:41.064797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:41.065246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:41.065654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:41.066520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] RBAC: Adding query filter params (interface=public, service_id=282aefc7f2f64e0192740d6de298d0dd, region_id=RegionOne) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:41.066993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:41.068619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f2071-6e06-4ef6-86af-267b096734e5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:41.075223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 389/771] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:12:40 2022] GET /identity/v3/endpoints?service_id=282aefc7f2f64e0192740d6de298d0dd&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:41.081830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:41.193819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d2576-23ee-44f8-98ba-e64cf73dfdba None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:41.221610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:41.230135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:41.230757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:41.231068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:41.231393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:41.232010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:41.233528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c2dbe-7b90-4583-bc54-10fbdb23315f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:41.240721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 383/772] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:12:41 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:12:41.897933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0a2a-1b29-415d-8a9d-242aabf97ff2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:41.898489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0a2a-1b29-415d-8a9d-242aabf97ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:41.898489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a0a2a-1b29-415d-8a9d-242aabf97ff2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:41.899813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 390/773] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:41.906530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a96c-429c-47c7-97c9-034176d808d4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:41.907082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a96c-429c-47c7-97c9-034176d808d4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:41.907082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5a96c-429c-47c7-97c9-034176d808d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:41.927552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-11f5a96c-429c-47c7-97c9-034176d808d4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:41.929086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11f5a96c-429c-47c7-97c9-034176d808d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:41.998856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 384/774] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:41 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:42.016403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b2d3-22e3-4ac1-b006-99e5866408f9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.016688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b2d3-22e3-4ac1-b006-99e5866408f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.016940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4b2d3-22e3-4ac1-b006-99e5866408f9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.018454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 391/775] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:42.023575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.023863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.024134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.042661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:42.043883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:42.113425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 385/776] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:42.262305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4b2d3-22e3-4ac1-b006-99e5866408f9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:42.275832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:42.373053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4b2d3-22e3-4ac1-b006-99e5866408f9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:42.378297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:42.407609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:42.408377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.408742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.409092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.410303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:42.411874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:42.413712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 392/777] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:42.419464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:42.443503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:42.499100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:42.501652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73210e3-f88c-49ff-bd71-b6aa40a4304c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:42.523187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948f691-5839-4281-a0f1-11eaf43e3631 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:42.536650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:42.537571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.537697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.537877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.538462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:42.539856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:42.548621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:42.549796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:42.550782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 386/778] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:42.556110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:42.563457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:42.564070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.564348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.564623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.565265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:42.566705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:42.596148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 393/779] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:12:42 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:42.679018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546a0f3-be96-4a4d-a0d9-23212d585475 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:42.697871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:42.720573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:42.721305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.721688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.722059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.722888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:42.723319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:42.724863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:42.735294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 387/780] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:42.737718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:42.845600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5d47-be04-4a7e-a6b2-86c7cce08faa None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:42.874929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:42.882546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:42.883461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:42.883846 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:42.884210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:42.884947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:42.885343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:42.887336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9474f-1424-4455-b63e-03409c1dcdc6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:42.897254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 394/781] 10.222.0.36 () {56 vars in 1112 bytes} [Tue Oct 25 13:12:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:42.898851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:43.004977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014104f-6f88-488c-a37f-df241a08b7b5 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:43.039098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:43.039758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:43.040036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:43.040307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:43.040980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:43.042355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255dc6-c7a7-4cc6-8487-cfbb67d523bc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:43.068829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 388/782] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:43.385807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c1be-6c4c-4bdf-93b8-359d2e44789e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:43.386378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c1be-6c4c-4bdf-93b8-359d2e44789e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:43.386378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c1be-6c4c-4bdf-93b8-359d2e44789e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:43.387925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 395/783] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:43.393163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097548-501d-4912-9663-d3583cae5b51 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:43.393522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097548-501d-4912-9663-d3583cae5b51 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:43.393875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63097548-501d-4912-9663-d3583cae5b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:43.413066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-63097548-501d-4912-9663-d3583cae5b51 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:43.414262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-63097548-501d-4912-9663-d3583cae5b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:43.445965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:43.493828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 389/784] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:43.756446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c1be-6c4c-4bdf-93b8-359d2e44789e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:43.862589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c1be-6c4c-4bdf-93b8-359d2e44789e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:43.897646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:43.898276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:43.898545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:43.898809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] PATH_INFO: `/v3/services/image` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:43.899418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:43.900803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:43.909861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:43.910844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 25 13:12:43.911714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:43.915604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 396/785] 10.222.0.36 () {56 vars in 1105 bytes} [Tue Oct 25 13:12:43 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:43.916944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63097548-501d-4912-9663-d3583cae5b51 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:44.018764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63097548-501d-4912-9663-d3583cae5b51 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:44.055086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:44.055744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:44.056014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:44.056267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:44.056906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] RBAC: Adding query filter params (name=image) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:44.057198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:44.058557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:44.063728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 390/786] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:44.069509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:44.127340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:44.170826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185bc1b-6961-4095-ad6c-686cf4da8d03 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:44.206130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:44.206735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:44.207005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:44.207298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:44.207922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:44.208216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:44.210198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f53e04-8105-4e8f-89d3-12da736491b9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:44.215247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 397/787] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:44.221077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:44.249040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:44.320985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c8b15-6ab5-4123-8bca-3a6e33bf8bdd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:44.356608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:44.357225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:44.357505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:44.357850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:44.358540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:44.359966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf638-5607-4c09-b561-646e4cb3eee0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:44.382152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 391/788] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:12:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:44.782873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecb05e-7fe6-47af-8c6a-654ec3dd7989 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:44.783442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecb05e-7fe6-47af-8c6a-654ec3dd7989 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:44.783442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecb05e-7fe6-47af-8c6a-654ec3dd7989 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:44.784960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 398/789] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:44.790220 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd676afb-06d9-45c4-ad05-a48096098e2d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:44.790514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd676afb-06d9-45c4-ad05-a48096098e2d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:44.790785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd676afb-06d9-45c4-ad05-a48096098e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:44.810470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dd676afb-06d9-45c4-ad05-a48096098e2d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:44.811775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd676afb-06d9-45c4-ad05-a48096098e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:44.891356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 392/790] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:45.156366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecb05e-7fe6-47af-8c6a-654ec3dd7989 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:45.264333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecb05e-7fe6-47af-8c6a-654ec3dd7989 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:45.280215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1dfb2-92bb-4630-8f73-97f8bcb67048 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.280774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1dfb2-92bb-4630-8f73-97f8bcb67048 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.280774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1dfb2-92bb-4630-8f73-97f8bcb67048 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.282387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 393/791] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:12:45.288103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.288364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.288648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.290670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:45.307780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:45.308198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:45.308858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.309031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:45.309127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.309382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.310493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:45.311177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:45.312018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:45.315187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 399/792] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:45.319410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:45.380187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 394/793] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:12:45.422579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99845199-a4fa-43d0-8620-71517e99440b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:45.457263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:45.458133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.458600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.459056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.459967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:45.461607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:45.485221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 400/794] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:12:45 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:12:45.643172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:45.746044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb5de0-e9fd-4d0b-bdb4-bf6c33267ced None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:45.779437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:45.780112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.780385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.780652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.787333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:45.792145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:45.797556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:45.798565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:45.799494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 395/795] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:12:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:45.806478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:45.854151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:45.909560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3935d9-ee53-4c87-bdff-08d792f624be None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:45.941920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:45.942531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:45.942806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:45.943072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:45.943769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:45.944065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:45.945432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:45.958770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 401/796] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Oct 25 13:12:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:45.960055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.061116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb3795-b07f-4e99-bba8-8e659fe60eb3 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.095504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.096217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.096417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.096629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.107094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.108685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:46.123521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:46.124603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:46.126110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:46.126110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:46.126110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:46.126110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 396/797] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:46.130288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.232788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419039-ab6c-4bbb-93d1-6c1af74394e7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.241968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:46.268307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.268907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.269182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.269450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.270091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:46.270390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.271815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.283896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 402/798] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:46.289479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.341341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:46.390155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb043-3c24-4596-88a8-11b75eb17536 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.423044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.423714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.423987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.424252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.434325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.436008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-59b850a0-471d-48e3-8e88-82eb7f8523f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:46.448847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:46.449922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:46.450877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:46.452131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 397/799] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:46.456634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.561718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa44cb-91d7-4400-acd6-4de9db6ea399 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.561989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa44cb-91d7-4400-acd6-4de9db6ea399 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.562254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa44cb-91d7-4400-acd6-4de9db6ea399 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.563659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 398/800] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:46.565530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a343a9c-9353-4047-a114-352d9cafb4d3 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.568587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.568863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.569131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.587857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:46.589103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:46.602778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.603349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.603633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.603891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.604617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:46.604920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.606351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.625576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 403/801] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:46.629573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.656084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:46.675477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 399/802] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:46.731376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6e9c-eab7-4e52-b74d-40a977c25278 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.767511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.768115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.768395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.768671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.778768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.780342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:46.794981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:46.796175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:46.802374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:46.802374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:46.802374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:46.802374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 404/803] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:46.803852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.847451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:46.906507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342354f0-fefb-4fe8-adf1-910674160927 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:46.938895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:46.940790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:46.941444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:46.941732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:46.942023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:46.942659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:46.942956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:46.944387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:46.956479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 400/804] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:46.962062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:47.027028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:47.050728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4e312-08c3-43bc-818c-af6fdab5f7c5 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.064432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:47.067248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddcc0e-f17e-485d-8629-0acb794d26a8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.089355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.090001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.090282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.090560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.091270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.092798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.101767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.sql.core [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:47.101767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:12:47.101767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [parameters: {'id': 'b3dfde0aa9b84d048b05146e0ab2d4a9', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 25 13:12:47.101767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81437) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 25 13:12:47.103181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.104035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.104480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.104918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:47.109400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 25 13:12:47.110215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application yield resource Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application next(self.gen) Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.rollback() Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.force_reraise() Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise self.value Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application session.commit() Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 25 13:12:47.111185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._prepare_impl() Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.session.flush() Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._flush(objects) Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application compat.raise_( Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application flush_context.execute() Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rec.execute(self) Oct 25 13:12:47.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 25 13:12:47.113057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result = self._query(query) Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application conn.query(q) Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application result.read() Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 25 13:12:47.113863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [parameters: {'id': 'b3dfde0aa9b84d048b05146e0ab2d4a9', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 25 13:12:47.114744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:47.115578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.116518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 405/805] 10.222.0.36 () {60 vars in 1137 bytes} [Tue Oct 25 13:12:46 2022] POST /identity/v3/domains => generated 190 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:47.117223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:47.118716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.119335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:47.133172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:47.134888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:47.136066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 401/806] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:47.140448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:47.225213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db07f-1827-4471-a65f-708e8eef74d0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.249598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302d4bc-745a-4539-bf14-a994b8cdd629 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.260175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.260814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.261095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.261364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.272497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.274190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.276301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:47.288561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.289179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.289453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.289779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.290531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:47.290879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:47.291513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:47.292420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:47.296288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:47.296288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:47.296288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:47.296506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.296591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 406/807] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:47.298893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:47.308112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 402/808] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:47.313935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:47.338484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:47.407623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e70eb2-76df-4e2e-bc87-896a2945c30e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.418166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93611-60a6-4421-96ff-0162683035a1 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:47.443361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.443973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.444248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.444524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.445152 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:47.445442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.446810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95926bd1-9ee5-49c9-a6b1-4e528db001e4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.456128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:47.456813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.457092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.457363 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/840f7fde7a13466c8804a7a41a340461` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.471466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 407/809] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:47.471689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=b45a976b6672412f92f9a4f8fbc75b1b, role_id=840f7fde7a13466c8804a7a41a340461)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:47.471689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a26f7b-0ff6-4c2a-9930-9c7413baa0f4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:47.486928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 403/810] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:47 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/b45a976b6672412f92f9a4f8fbc75b1b/roles/840f7fde7a13466c8804a7a41a340461 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:12:47.698466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24748bf-f09a-4d14-ae4e-a8c3e7aa2c38 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.698796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24748bf-f09a-4d14-ae4e-a8c3e7aa2c38 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.698988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24748bf-f09a-4d14-ae4e-a8c3e7aa2c38 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.700388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 408/811] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:47.706332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:47.706623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:47.706893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:47.725957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:47.727137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:47.802951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 404/812] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:48.065719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24748bf-f09a-4d14-ae4e-a8c3e7aa2c38 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:48.121556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:48.167606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24748bf-f09a-4d14-ae4e-a8c3e7aa2c38 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:48.205029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:48.205665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:48.205943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:48.206211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:48.216972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:48.221797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:48.222897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:48.223924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 409/813] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:48.229050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:48.240761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:48.330260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e6626-7b14-4c3d-9eab-3b5ec0a3c2c8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:48.365608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:48.366230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:48.366505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:48.366815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:48.367515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:48.367821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:48.369197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:48.377454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 405/814] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:48.383414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:48.485684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c30145-d5c7-4772-98ca-e3fdcf02aef7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:48.499722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:48.521377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:48.522002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:48.522284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:48.522551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] PATH_INFO: `/v3/users/3c605bcf972b4817b486bb5f19d29a85` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:48.535379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] RBAC: Authorizing `identity:get_user(user_id=3c605bcf972b4817b486bb5f19d29a85)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:48.536828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:48.538462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 410/815] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:12:48 2022] GET /identity/v3/users/3c605bcf972b4817b486bb5f19d29a85 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:48.544376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:48.619970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:48.651053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b95cc-5ef8-40a5-92b2-e78bdfa65c31 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:48.685031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:48.685724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:48.686003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:48.686270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:48.687443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] RBAC: Authorizing `identity:get_project(project_id=b00c9d1b08834b7cbe9089faa8ded5e6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:48.688880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d4ad4-9a66-4216-9f26-d85fa28f49bc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:48.690555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 406/816] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:12:48 2022] GET /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:48.696599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:48.794953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3a53-e745-4128-ad3f-35cf5477357a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:48.830162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:48.830818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:48.831089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:48.831349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6/users/3c605bcf972b4817b486bb5f19d29a85/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:48.833356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] RBAC: Authorizing `identity:create_grant(project_id=b00c9d1b08834b7cbe9089faa8ded5e6, user_id=3c605bcf972b4817b486bb5f19d29a85, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:48.834808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5007f-57d7-4299-b58a-e0f94598ca6d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:48.849178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 411/817] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:48 2022] PUT /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6/users/3c605bcf972b4817b486bb5f19d29a85/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:12:49.661116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c008-9ee0-4138-9126-0b5caab20121 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:49.661646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c008-9ee0-4138-9126-0b5caab20121 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:49.661646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c008-9ee0-4138-9126-0b5caab20121 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:49.663022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 407/818] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:49.668603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:49.668995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:49.669143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:49.688176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:49.689401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:49.718473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:49.762918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 412/819] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:50.029684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094c008-9ee0-4138-9126-0b5caab20121 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.136899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094c008-9ee0-4138-9126-0b5caab20121 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:50.171274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:50.171900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.172183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.172445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.180157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:50.183173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:50.188696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:50.189778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:50.190625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 408/820] 10.222.0.36 () {56 vars in 1120 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:50.197078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.300023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0224d-b291-47fc-a1c4-c6c601b1a1bf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:50.334824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:50.335478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.335723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.336012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.336762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:50.337131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:50.338652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:50.346968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 413/821] 10.222.0.36 () {56 vars in 1115 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:50.353150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.458427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47452c-57a0-445b-942b-53fdbc7b4ded None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:50.494200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:50.494843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.495118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.495398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.505781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:50.507274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:50.641881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:50.642968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:50.644040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:50.644040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:50.644040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:50.644040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 409/822] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:50.649114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.676953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:50.757101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07355e-9d1b-48ba-b178-1f55e37f513d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:50.793973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:50.794595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.794873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.795169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.795824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:50.796126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:50.797503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e3dbd8-870d-4d70-ab8b-34fb95389c7e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:50.809325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 414/823] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:50.815547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.875675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:50.923324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce04e62-eaac-42bd-bf00-bd394cbdb396 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:50.958078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:50.958706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.958983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.959085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54e4a-8cc0-4e61-87e1-69aa0e784923 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.959314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.959453 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54e4a-8cc0-4e61-87e1-69aa0e784923 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.959708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54e4a-8cc0-4e61-87e1-69aa0e784923 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.961029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 415/824] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:50.965945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:50.966227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:50.966492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:50.966626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:50.968065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:12:50.979553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:50.980601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:50.981574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 410/825] 10.222.0.36 () {56 vars in 1093 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:50.985272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:50.986427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:50.986561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:51.053204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 416/826] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:51.086529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17b3a1-3a2c-486e-a3d4-a21bbe34e72f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.099113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.120023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.120634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.120907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.121173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.121895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:51.122188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.123681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.140143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 411/827] 10.222.0.36 () {56 vars in 1133 bytes} [Tue Oct 25 13:12:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:51.146067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.162116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.249088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb0f6c-5d28-42b3-8424-8e2aa5f3ef59 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.282874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.283507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.283765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.284030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.322280 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.349988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.351524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.365544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.421973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d36181-5671-4a31-b7ef-0626e78c04f8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.451125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 417/828] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:12:51.455931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.456564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.456838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.457102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.457347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.468032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:12:51.472814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:51.474081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:12:51.475290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 412/829] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:51.479824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.512858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.560031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19ceb2-d70a-49c0-b634-098ef1824506 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.574484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.585843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1347-ac51-472d-8171-38c14cf5742f None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.596190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.596825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.597101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.597368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.607375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.608844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.619931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.620547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.620830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.621100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.621769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:51.622069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:12:51.623439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:51.624385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:12:51.625279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:12:51.625279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:12:51.625279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:51.625451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.625521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 418/830] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:12:51.630269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.638176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 413/831] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:51.644044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.730061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ee9a-9dc3-4898-8198-a9db29eb8cde None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.731812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.741130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615270f9-f1b6-4860-a95e-f63875999d1d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.763004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.763583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.763896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.764139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.764814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:51.765110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.766547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.774797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.775466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.775722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.775987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] PATH_INFO: `/v3/users/3c605bcf972b4817b486bb5f19d29a85` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.777091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] RBAC: Authorizing `identity:get_user(user_id=3c605bcf972b4817b486bb5f19d29a85)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.778071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 419/832] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:51.779796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.781424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 414/833] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/users/3c605bcf972b4817b486bb5f19d29a85 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:51.783940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.786835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.811861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.883880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd688e-5584-4b2b-aae6-d02331380489 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.886192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d0a2-6a19-4167-b028-645e085d0743 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:51.916946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.917571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.917848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.918117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.919313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:51.920025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:51.920264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:51.920526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:51.924681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.926120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:12:51.937226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:12:51.938301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:12:51.939350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 420/834] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:12:51.944118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:51.949019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:51.989937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:51.991438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:52.025737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:52.040415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90257393-f8b7-4687-9703-51b0622fe60b None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:52.072943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:52.073547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:52.073827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:52.074100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:52.074796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:52.075101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:52.078326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:52.086255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 415/835] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:12:52.091889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 421/836] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:52.092198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:52.097657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:52.151416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:52.190952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16af8f-3c04-4826-a06b-8594438b65ad None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:52.192137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383cde0-ceb7-4354-b99c-03341c1c3274 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:52.226461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:52.227060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:52.227351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:52.227451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:52.227615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:52.228016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:52.228287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:52.228381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] RBAC: Adding query filter params (role.id=840f7fde7a13466c8804a7a41a340461, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=b45a976b6672412f92f9a4f8fbc75b1b) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:52.228555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:52.228648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:52.229683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b00c9d1b08834b7cbe9089faa8ded5e6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:52.230047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347683a7-9ecf-443f-bb2f-c4a9eb38fc29 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:52.231118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:52.238213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 416/837] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:12:52 2022] GET /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:52.238363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 422/838] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:52 2022] GET /identity/v3/role_assignments?user.id=b45a976b6672412f92f9a4f8fbc75b1b&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=840f7fde7a13466c8804a7a41a340461 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:52.239314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:52.354627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6322e-89bf-4ba9-9506-7e17e826afb8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:52.384125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:52.393628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:52.394251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:52.394542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:52.394829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:52.395585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=b00c9d1b08834b7cbe9089faa8ded5e6, user.id=3c605bcf972b4817b486bb5f19d29a85) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:12:52.395891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:52.397340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad494a-b74a-4d97-8635-4b628517d6a5 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:52.404047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 417/839] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:12:52 2022] GET /identity/v3/role_assignments?user.id=3c605bcf972b4817b486bb5f19d29a85&scope.project.id=b00c9d1b08834b7cbe9089faa8ded5e6&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:54.555903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999838-b3eb-4953-9d9f-359c1156b37a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:54.556750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999838-b3eb-4953-9d9f-359c1156b37a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:54.556750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999838-b3eb-4953-9d9f-359c1156b37a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:54.558831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 423/840] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:54.564599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:54.565036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:54.565137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:54.584714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:54.585910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:54.654796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 418/841] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:54.927534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999838-b3eb-4953-9d9f-359c1156b37a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:54.948444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:55.053240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999838-b3eb-4953-9d9f-359c1156b37a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:55.089348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:55.089997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:55.090279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:55.090551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:55.091829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:55.093251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34686077-a0ed-4851-97e2-d315409bdd7d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:55.094822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 424/842] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:12:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:55.100938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:55.212224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34344-5b80-4f4b-81dd-109e0b0bc845 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:55.235701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:55.248829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:55.249437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:55.249714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:55.249983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:55.250630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:55.252106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6838b4-5699-408c-b926-e0705229fba3 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:55.278673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 419/843] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:12:55 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:12:57.397103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278dc4a-83a2-4573-95d8-b8d56d916f18 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:57.397692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278dc4a-83a2-4573-95d8-b8d56d916f18 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:57.397692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278dc4a-83a2-4573-95d8-b8d56d916f18 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:57.398982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 425/844] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:12:57.405372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:57.406044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:57.406181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:57.430520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:12:57.431736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:12:57.502458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 420/845] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:12:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:12:57.768352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278dc4a-83a2-4573-95d8-b8d56d916f18 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:57.867203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:57.893431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278dc4a-83a2-4573-95d8-b8d56d916f18 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:57.928781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:57.929370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:57.929675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:57.929941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] PATH_INFO: `/v3/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:57.931470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] RBAC: Authorizing `identity:get_role(role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:57.932841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:57.934149 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 426/846] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:12:57 2022] GET /identity/v3/roles/45f94767c7a04887a34b82d23eb15d91 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:57.940287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:58.057261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df1f6-1a73-4d93-b0b6-d81d882b67b6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:58.066291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:12:58.093587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:58.094207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:58.094480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:58.094744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] PATH_INFO: `/v3/users/7348ecf683c84360af373da9c7a6c647` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:58.108117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] RBAC: Authorizing `identity:get_user(user_id=7348ecf683c84360af373da9c7a6c647)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:58.109548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:58.111117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 421/847] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:12:57 2022] GET /identity/v3/users/7348ecf683c84360af373da9c7a6c647 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:12:58.120053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:58.228319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396eb12f-7923-42c6-b336-cb5ad8075d18 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:58.266092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:58.266707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:58.266981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:58.267279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:58.268507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] RBAC: Authorizing `identity:get_project(project_id=b00c9d1b08834b7cbe9089faa8ded5e6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:58.269937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019c7fb-b7da-4df0-a8b9-cdf339ab3b3b None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:58.271593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 427/848] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:12:58 2022] GET /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:12:58.277922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:12:58.383197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d883b1-d056-4c68-af32-f29210e6444d None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:12:58.418084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:12:58.418689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:12:58.418994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:12:58.419292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6/users/7348ecf683c84360af373da9c7a6c647/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:12:58.421645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00c9d1b08834b7cbe9089faa8ded5e6, user_id=7348ecf683c84360af373da9c7a6c647, role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:12:58.423077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609b29d-a3fa-4fbf-bb37-5cf8a318f0f8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:12:58.440423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 422/849] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:12:58 2022] PUT /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6/users/7348ecf683c84360af373da9c7a6c647/roles/45f94767c7a04887a34b82d23eb15d91 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:13:00.569619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b164a-3d29-4bc5-9eeb-a36a4391fc7d None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:00.570460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b164a-3d29-4bc5-9eeb-a36a4391fc7d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:00.570460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b164a-3d29-4bc5-9eeb-a36a4391fc7d None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:00.572554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 428/850] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:00.579050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:00.579514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:00.579636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:00.603906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:00.605098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:00.626790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:00.676915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 423/851] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:00.943679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b164a-3d29-4bc5-9eeb-a36a4391fc7d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:00.962314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:01.073899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b164a-3d29-4bc5-9eeb-a36a4391fc7d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:01.110532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:01.111165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:01.111472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:01.111735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] PATH_INFO: `/v3/roles/45f94767c7a04887a34b82d23eb15d91` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:01.113191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] RBAC: Authorizing `identity:get_role(role_id=45f94767c7a04887a34b82d23eb15d91)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:01.114571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:01.115931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 429/852] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:00 2022] GET /identity/v3/roles/45f94767c7a04887a34b82d23eb15d91 => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:01.121917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:01.223020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98062ec-7792-47dd-a3a5-eb0d4247bc88 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:01.258222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:01.258847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:01.259126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:01.259426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] PATH_INFO: `/v3/users/7348ecf683c84360af373da9c7a6c647` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:01.260678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] RBAC: Authorizing `identity:get_user(user_id=7348ecf683c84360af373da9c7a6c647)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:01.262111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:01.263725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 424/853] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:01 2022] GET /identity/v3/users/7348ecf683c84360af373da9c7a6c647 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:01.270280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:01.357515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:01.376993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c1003-8ed0-45e9-a506-b13a435f60d9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:01.412942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:01.413538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:01.413813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:01.414083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:01.610759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:01.612240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:01.695019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:01.713330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 430/854] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:13:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:13:01.719439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:01.746390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:01.824488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2007b6-5f64-45a5-a72f-e7d99e66591e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:01.859893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:01.860511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:01.860845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:01.861024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] PATH_INFO: `/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:01.862300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] RBAC: Authorizing `identity:get_project(project_id=b00c9d1b08834b7cbe9089faa8ded5e6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:01.863765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ee2a8-8785-4783-afe0-c5dfb55dbd54 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:01.865378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 425/855] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:13:01 2022] GET /identity/v3/projects/b00c9d1b08834b7cbe9089faa8ded5e6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:01.874808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:01.974724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d69fa-92e9-4a14-90f2-2a8a3fa9d9de None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:02.010237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:02.010821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:02.011097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:02.011394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:02.012114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] RBAC: Adding query filter params (role.id=45f94767c7a04887a34b82d23eb15d91, scope.project.id=b00c9d1b08834b7cbe9089faa8ded5e6, user.id=7348ecf683c84360af373da9c7a6c647) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:02.012422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:02.013865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926198f-4d9d-4a4e-85cb-a0919ac5562c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:02.019674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 431/856] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:13:01 2022] GET /identity/v3/role_assignments?user.id=7348ecf683c84360af373da9c7a6c647&scope.project.id=b00c9d1b08834b7cbe9089faa8ded5e6&role.id=45f94767c7a04887a34b82d23eb15d91 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:13:04.122411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745984b-213b-4d55-a0bd-9e31080cff94 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:04.123381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745984b-213b-4d55-a0bd-9e31080cff94 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:04.123381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745984b-213b-4d55-a0bd-9e31080cff94 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:04.125466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 426/857] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:04.131826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:04.132595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:04.132754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:04.155605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:04.156793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:04.202821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:04.234604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 432/858] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:04.505764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745984b-213b-4d55-a0bd-9e31080cff94 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:04.609158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:04.621638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745984b-213b-4d55-a0bd-9e31080cff94 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:04.658164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:04.658786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:04.659065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:04.659370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:04.660528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:04.661985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823ffee-48fc-4da9-8f47-ca65d551a37f None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:04.663599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 427/859] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:04.669736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:04.771009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cd81a-9a3a-45bc-ad04-6fd8082c2805 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:04.805094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:04.805682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:04.805954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:04.806216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:04.806859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:04.808289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d5bde-427d-496b-b53f-b2bfb82d1507 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:04.836295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 433/860] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:13:04 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:13:06.985715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0164-6b5f-45e0-ae42-9c938be98554 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:06.986224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0164-6b5f-45e0-ae42-9c938be98554 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:06.986224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0164-6b5f-45e0-ae42-9c938be98554 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:06.988225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 428/861] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:06.997164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:06.997559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:06.997779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:07.023752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:07.024938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:07.050329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:07.098016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 434/862] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:07.364899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0164-6b5f-45e0-ae42-9c938be98554 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:07.466101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0164-6b5f-45e0-ae42-9c938be98554 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:07.500784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:07.501410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:07.501661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:07.501951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:07.503096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:07.504553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca3651-1548-4899-9597-ece4c1fc8c68 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:07.506118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 429/863] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:13:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:07.512125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:07.630235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefeecb-dcb0-4ce4-9578-43b92306661b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:07.668314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:07.668914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:07.669188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:07.669455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:07.670115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:07.671589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725013-68da-4c79-b467-0a4c17473bdd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:07.698500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 435/864] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:13:07 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:13:09.860717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eedee-5677-46b0-9aee-ccf6ff6dad24 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:09.861300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eedee-5677-46b0-9aee-ccf6ff6dad24 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:09.861300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eedee-5677-46b0-9aee-ccf6ff6dad24 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:09.862628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 430/865] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:09.869035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:09.869482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:09.869582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:09.895723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:09.896865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:09.906667 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:09.970126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 436/866] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:10.235237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742eedee-5677-46b0-9aee-ccf6ff6dad24 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:10.266349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:10.366147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742eedee-5677-46b0-9aee-ccf6ff6dad24 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:10.405282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:10.405904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:10.406219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:10.406494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:10.416917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:13:10.421769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:13:10.422862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:13:10.424216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 431/867] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:13:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:13:10.432614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:10.538393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03af3a-0164-4c79-8b7e-0026fab62340 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:10.573536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:10.574130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:10.574409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:10.574678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:10.575377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:10.575688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:10.577054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:10.585582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 437/868] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:13:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:10.592350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:10.690997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:10.705271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f26187-c751-489f-a182-2a249a56b451 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:10.739535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:10.740155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:10.740432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:10.740704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] PATH_INFO: `/v3/users/0ddcbd83fea9447c9419fda981fae3b8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:10.754107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] RBAC: Authorizing `identity:get_user(user_id=0ddcbd83fea9447c9419fda981fae3b8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:10.755614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:10.757216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 432/869] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:10 2022] GET /identity/v3/users/0ddcbd83fea9447c9419fda981fae3b8 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:10.763534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:10.772995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:10.868830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e833a6-fda6-4d28-ac7c-27a80dde29a9 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:10.903460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:10.904050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:10.904310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:10.904608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] PATH_INFO: `/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:10.905861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] RBAC: Authorizing `identity:get_project(project_id=c571cc47cabf4b54aeb1146d324a6f0b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:10.907361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d961f-13c7-4309-bcbc-85dea1a3c808 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:10.909005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 438/870] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:13:10 2022] GET /identity/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:10.915356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:11.020531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a652b6-2c74-43b2-8789-dbcf45862c98 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:11.066090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:11.066743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:11.067019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:11.067321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] PATH_INFO: `/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b/users/0ddcbd83fea9447c9419fda981fae3b8/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:11.069379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c571cc47cabf4b54aeb1146d324a6f0b, user_id=0ddcbd83fea9447c9419fda981fae3b8, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:11.070827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676511c-6951-4d54-b30e-85363f8698a2 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:11.086430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 433/871] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:13:10 2022] PUT /identity/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b/users/0ddcbd83fea9447c9419fda981fae3b8/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:13:13.184023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aad79-de42-4e1d-8a07-a730a4e1c82b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:13.184929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aad79-de42-4e1d-8a07-a730a4e1c82b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:13.184929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769aad79-de42-4e1d-8a07-a730a4e1c82b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:13.187042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 439/872] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:13.192912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:13.193371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:13.193479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:13.208243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:13.214463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:13.216371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:13.283942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 434/873] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:13:13.547231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aad79-de42-4e1d-8a07-a730a4e1c82b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:13.647875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:13.674831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769aad79-de42-4e1d-8a07-a730a4e1c82b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:13.709884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:13.710494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:13.710766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:13.711031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:13.724474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:13:13.729203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:13:13.730201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:13:13.731201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 440/874] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:13:13.737626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:13.848762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec2835-4bae-4701-a749-d742caed8363 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:13.885001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:13.885597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:13.885875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:13.886146 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:13.886814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:13.887118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:13.888527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:13.890759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:13.897789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 435/875] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:13.904590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:14.007689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d93da5-2f50-4b75-8e76-f1d3b8e7cd88 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:14.042427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:14.043029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:14.043330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:14.043598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] PATH_INFO: `/v3/users/0ddcbd83fea9447c9419fda981fae3b8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:14.044820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] RBAC: Authorizing `identity:get_user(user_id=0ddcbd83fea9447c9419fda981fae3b8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:14.046238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:14.047847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 441/876] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:13 2022] GET /identity/v3/users/0ddcbd83fea9447c9419fda981fae3b8 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:14.056999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:14.170476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652caa07-9fea-46c5-b754-8a2bf5d67ad7 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:14.206217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:14.206794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:14.207077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:14.207382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:14.271922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:14.276706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:14.279072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:14.386927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 436/877] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:13:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:13:14.393885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:14.397364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:14.499757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d3523-3a7d-4cb1-be0c-a5a8c1318280 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:14.533198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:14.533818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:14.534086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:14.534350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] PATH_INFO: `/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:14.535535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] RBAC: Authorizing `identity:get_project(project_id=c571cc47cabf4b54aeb1146d324a6f0b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:14.536953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103124e-f8e6-4997-b0d7-3b640042066c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:14.538559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 442/878] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:13:14 2022] GET /identity/v3/projects/c571cc47cabf4b54aeb1146d324a6f0b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:14.545017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:14.642565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:14.646537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111670db-2eea-4913-a824-e6a71bac3052 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:14.681593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:14.682184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:14.682453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:14.682717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:14.683468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=c571cc47cabf4b54aeb1146d324a6f0b, user.id=0ddcbd83fea9447c9419fda981fae3b8) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:14.683769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:14.685195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa82e3-82a5-45dc-a91d-0e2a9e46ade0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:14.691065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 437/879] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:13:14 2022] GET /identity/v3/role_assignments?user.id=0ddcbd83fea9447c9419fda981fae3b8&scope.project.id=c571cc47cabf4b54aeb1146d324a6f0b&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:16.828687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31773a73-357c-403d-b90d-8a31ccc0d30e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:16.829574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31773a73-357c-403d-b90d-8a31ccc0d30e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:16.829574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31773a73-357c-403d-b90d-8a31ccc0d30e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:16.832183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 443/880] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:16.838534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f84d8f-acff-4e5f-8556-2203793efbe7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:16.839270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f84d8f-acff-4e5f-8556-2203793efbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:16.839480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f84d8f-acff-4e5f-8556-2203793efbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:16.865927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d9f84d8f-acff-4e5f-8556-2203793efbe7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:16.867285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d9f84d8f-acff-4e5f-8556-2203793efbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:16.947653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 438/881] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:17.221387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31773a73-357c-403d-b90d-8a31ccc0d30e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:17.309072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:17.349525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31773a73-357c-403d-b90d-8a31ccc0d30e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:17.384263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:17.384855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:17.385130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:17.385407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:17.386079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:17.387482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27647b76-cfd6-4438-b9fa-c0b432899bdd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:17.406731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 444/882] 10.222.0.36 () {60 vars in 1138 bytes} [Tue Oct 25 13:13:17 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:13:19.565379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c17f-cf1e-4829-b304-3a1fea6f1ca5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:19.566283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c17f-cf1e-4829-b304-3a1fea6f1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:19.566283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628c17f-cf1e-4829-b304-3a1fea6f1ca5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:19.568409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 439/883] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:19.577325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:19.577802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:19.577955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:19.597725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:19.598900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:19.683465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 445/884] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:19.949263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628c17f-cf1e-4829-b304-3a1fea6f1ca5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:20.051634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:20.062116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628c17f-cf1e-4829-b304-3a1fea6f1ca5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:20.096680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:20.097288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:20.097559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:20.097825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] PATH_INFO: `/v3/domains/67a55eb7aaf6408aaeec59f2d85d886c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:20.108516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=67a55eb7aaf6408aaeec59f2d85d886c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:20.109971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbd615-6d6c-4947-bb8f-5113f4c7cb2d None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:20.111549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 440/885] 10.222.0.36 () {56 vars in 1183 bytes} [Tue Oct 25 13:13:19 2022] GET /identity/v3/domains/67a55eb7aaf6408aaeec59f2d85d886c => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:20.121607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:20.182254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:20.238032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97965c6-c09a-4c97-9459-8fb7914f0179 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:20.272484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:20.273077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:20.273352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:20.273580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:20.274255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:20.275720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99dabc-f86c-4f47-ad08-64c59b4d421c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:20.305322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 446/886] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:13:20 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:13:22.477170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0388-f68f-48e1-a68e-a19fe113736d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:22.478059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0388-f68f-48e1-a68e-a19fe113736d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:22.478059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0388-f68f-48e1-a68e-a19fe113736d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:22.480213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 441/887] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:22.485753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:22.486194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:22.486291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:22.506139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:22.507359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:22.577930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 447/888] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:22.843317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d0388-f68f-48e1-a68e-a19fe113736d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:22.955789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d0388-f68f-48e1-a68e-a19fe113736d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:22.990883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:22.991517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:22.991793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:22.992058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] PATH_INFO: `/v3/domains/67a55eb7aaf6408aaeec59f2d85d886c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:22.993192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=67a55eb7aaf6408aaeec59f2d85d886c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:22.994617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe384a-5dc3-462d-aa02-bf8e11554037 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:22.996199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 442/889] 10.222.0.36 () {56 vars in 1183 bytes} [Tue Oct 25 13:13:22 2022] GET /identity/v3/domains/67a55eb7aaf6408aaeec59f2d85d886c => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:23.003621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:23.050383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:23.112877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9acb42-3b10-4219-9cf8-e18bfca343e9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:23.151124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:23.151754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:23.152051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:23.152298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:23.152948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:23.154393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd4285-ee9b-4fec-8843-0359b9f64c3d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:23.180716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 448/890] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:13:23 2022] POST /identity/v3/users => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:13:25.362759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f15b8-e0da-4857-bbad-ea3df3a60e34 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:25.363694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f15b8-e0da-4857-bbad-ea3df3a60e34 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:25.363694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f15b8-e0da-4857-bbad-ea3df3a60e34 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:25.365765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 443/891] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:25.371262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:25.371677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:25.371788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:25.390416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:25.391637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:25.460010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 449/892] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:25.729413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f15b8-e0da-4857-bbad-ea3df3a60e34 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:25.757037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:25.840056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f15b8-e0da-4857-bbad-ea3df3a60e34 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:25.878459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:25.879038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:25.879344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:25.879608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:25.889895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:13:25.894669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:13:25.895881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:25.896981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:13:25.899321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 444/893] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:13:25.901750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:25.930003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:26.009146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564423ac-3018-46cb-9451-7cc7d4deb858 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:26.044133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:26.044730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:26.045007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:26.045282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:26.045953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:26.046252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:26.047701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:26.055573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 450/894] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:26.064581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:26.159443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:26.179791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b28e-cac3-4d8b-b868-23eea9727d1c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:26.215027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:26.215666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:26.215951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:26.216204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] PATH_INFO: `/v3/users/c2573276df7c4d0aaa8f0f2a7d2a6928` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:26.230967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] RBAC: Authorizing `identity:get_user(user_id=c2573276df7c4d0aaa8f0f2a7d2a6928)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:26.232439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:26.234036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 445/895] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:26 2022] GET /identity/v3/users/c2573276df7c4d0aaa8f0f2a7d2a6928 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:26.240130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:26.337204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:26.349402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee2a8-bb26-4dbd-acd2-d104f2daffe4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:26.388469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:26.389072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:26.389361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:26.389641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] PATH_INFO: `/v3/projects/e188c85e02594615860d824eadc4baa8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:26.390921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] RBAC: Authorizing `identity:get_project(project_id=e188c85e02594615860d824eadc4baa8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:26.392369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c8c58-5aee-4077-a057-ec81c3597c5d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:26.394010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 451/896] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:13:26 2022] GET /identity/v3/projects/e188c85e02594615860d824eadc4baa8 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:26.401179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:26.509026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e931-a5d7-42e6-9949-a7406b8544d8 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:26.543744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:26.544384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:26.544655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:26.544923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] PATH_INFO: `/v3/projects/e188c85e02594615860d824eadc4baa8/users/c2573276df7c4d0aaa8f0f2a7d2a6928/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:26.547200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e188c85e02594615860d824eadc4baa8, user_id=c2573276df7c4d0aaa8f0f2a7d2a6928, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:26.548623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa111c1-ed4c-4873-a5e9-c3ad0b81cfa9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:26.558833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:26.564827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 446/897] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:13:26 2022] PUT /identity/v3/projects/e188c85e02594615860d824eadc4baa8/users/c2573276df7c4d0aaa8f0f2a7d2a6928/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:13:28.684105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032d1ea-5950-45cb-b900-2eefa12440ba None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:28.684966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032d1ea-5950-45cb-b900-2eefa12440ba None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:28.684966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032d1ea-5950-45cb-b900-2eefa12440ba None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:28.687097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 452/898] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:13:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:13:28.695922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:28.696367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:28.696474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:28.719802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:13:28.720992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:13:28.791937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 447/899] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:13:29.057348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032d1ea-5950-45cb-b900-2eefa12440ba None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:29.167452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032d1ea-5950-45cb-b900-2eefa12440ba None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:29.202437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:29.203079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:29.203390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:29.203666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:29.215102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:13:29.219995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:13:29.221033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:13:29.222268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 453/900] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:13:29.227276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:29.326378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5048903-d617-4bf2-a618-aa97ff6a6070 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:29.354159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:29.362907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:29.363547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:29.363826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:29.364090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:29.364754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:29.365052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:29.366412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:29.374670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 448/901] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:13:29.380763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:29.401726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:29.493501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369258a-40e8-4f79-870b-75b8a09ffa05 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:29.528168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045d6-a235-414f-abff-442826aff74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:29.528778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045d6-a235-414f-abff-442826aff74f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:29.529053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045d6-a235-414f-abff-442826aff74f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:29.529327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045d6-a235-414f-abff-442826aff74f None admin] PATH_INFO: `/v3/users/c2573276df7c4d0aaa8f0f2a7d2a6928` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:29.530573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69045d6-a235-414f-abff-442826aff74f None admin] RBAC: Authorizing `identity:get_user(user_id=c2573276df7c4d0aaa8f0f2a7d2a6928)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:29.532066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69045d6-a235-414f-abff-442826aff74f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:29.533629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 454/902] 10.222.0.36 () {56 vars in 1177 bytes} [Tue Oct 25 13:13:29 2022] GET /identity/v3/users/c2573276df7c4d0aaa8f0f2a7d2a6928 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:29.539638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:29.648309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8175d-9fc0-4c0e-81fe-438bf1ac8f2a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:29.683846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:29.684448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:29.684718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:29.684984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:29.746608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:29.758913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:29.760505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:29.873177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 449/903] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:13:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:13:29.879743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045d6-a235-414f-abff-442826aff74f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:29.971053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:29.986646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045d6-a235-414f-abff-442826aff74f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:30.021267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:30.021877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:30.022168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:30.022421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] PATH_INFO: `/v3/projects/e188c85e02594615860d824eadc4baa8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:30.023627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] RBAC: Authorizing `identity:get_project(project_id=e188c85e02594615860d824eadc4baa8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:30.025062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f126e-c4db-4f93-87ef-ac18c350cf0e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:30.026702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 455/904] 10.222.0.36 () {56 vars in 1186 bytes} [Tue Oct 25 13:13:29 2022] GET /identity/v3/projects/e188c85e02594615860d824eadc4baa8 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:30.033293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:13:30.136013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:30.147877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f48f-5cf5-4fce-b810-a6a628fe003a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:13:30.185408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:13:30.185997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:13:30.186271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:13:30.186534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:13:30.187288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=e188c85e02594615860d824eadc4baa8, user.id=c2573276df7c4d0aaa8f0f2a7d2a6928) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:13:30.187599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:13:30.189010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eeb403-dbe7-4396-a584-d3ecdc1982b0 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:13:30.194862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 450/905] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:13:30 2022] GET /identity/v3/role_assignments?user.id=c2573276df7c4d0aaa8f0f2a7d2a6928&scope.project.id=e188c85e02594615860d824eadc4baa8&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:13:40.016398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=23 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:13:40.192183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=23 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:25.265403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61fd0c-f3f1-4688-a6ad-129252a264f6 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:25.265403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61fd0c-f3f1-4688-a6ad-129252a264f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:25.265403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61fd0c-f3f1-4688-a6ad-129252a264f6 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:25.265403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 456/906] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:25.276441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:25.276441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:25.276441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:25.303198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:25.304042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:25.403004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 451/907] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:25.700581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61fd0c-f3f1-4688-a6ad-129252a264f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:25.826287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61fd0c-f3f1-4688-a6ad-129252a264f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:25.864668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:25.865383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:25.865677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:25.866055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:25.879475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:25.880909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:25.896033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:25.897125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:25.898098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:25.898098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:25.898098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:25.898098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 457/908] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:25.903531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:26.036421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22647033-036e-45b5-9c06-63faea58ecb2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:26.075718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:26.081519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:26.082393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:26.082881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:26.083279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:26.084076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:26.084437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:26.085916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f438f7-d7b6-4718-a5f4-b8362ae4cbae None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:26.103533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 452/909] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:14:26.111762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:26.180638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:26.243680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e589c-8321-4936-9c5b-62138fc92d2e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:26.292330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:26.293041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:26.293312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:26.293548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:26.294200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:26.295797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7ac9d-ab0c-47b3-90f0-8b6ce2b6c1e3 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:26.336004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 458/910] 10.222.0.36 () {60 vars in 1132 bytes} [Tue Oct 25 13:14:26 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:14:28.787847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00936-7ce4-43c8-be2a-341b84949034 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:28.788963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00936-7ce4-43c8-be2a-341b84949034 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:28.788963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00936-7ce4-43c8-be2a-341b84949034 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:28.789903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 453/911] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:14:28.799274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:28.799868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:28.799868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:28.819753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:28.820891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:28.899371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 459/912] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:29.164387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db00936-7ce4-43c8-be2a-341b84949034 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:29.274921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db00936-7ce4-43c8-be2a-341b84949034 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:29.314897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:29.315584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:29.315880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:29.316206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:29.329193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:14:29.335311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:29.337717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:29.338775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 454/913] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:29.343016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:29.397737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:29.470805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23264c91-d75e-4181-ae5b-7b614a56bc31 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:29.511941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:29.512566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:29.512874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:29.513210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:29.513997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] RBAC: Adding query filter params (name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:29.514306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:29.516180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:29.525693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 460/914] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:14:29.533306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:29.594047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:29.662676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c430a1-3e72-4df1-8f22-bd115048f886 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:29.706842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:29.707680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:29.708229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:29.708482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:29.720311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:29.721897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:29.739845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:29.741025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:29.742255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:29.742255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:29.742255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:29.742255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 455/915] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:29.748469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:29.866034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:29.870289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c4908-45bb-4e86-9d6f-50cd78b7392a None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:29.912482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:29.913157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:29.913460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:29.913717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:29.914451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:29.914864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:29.916545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:29.930240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 461/916] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:29.936787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:30.059441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2296c-091c-4275-a3ab-e527e1fb0823 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:30.060968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:30.099714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:30.100349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:30.100628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:30.100895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:30.112868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:30.114797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:14:30.129139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:30.132909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:30.133886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 456/917] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:30.136512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:30.247029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e0ef2-093f-4366-bd62-86868e15cbd7 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:30.282399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:30.283005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:30.283307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:30.283644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:30.284297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:30.284596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:30.286025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:30.305718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 462/918] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Oct 25 13:14:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:30.311587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:30.432224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77f84d-6289-44bb-829e-9d733676137c None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:30.471296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:30.471985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:30.472395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:30.472854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:30.486621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:30.488133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:30.504045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:30.511644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:30.512813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:30.512813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:30.512813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:30.512813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 457/919] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 25 13:14:30.513101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:30.537879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:30.640658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f93e0-e434-42ab-804b-84ca266f205d None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:30.688521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:30.689310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:30.689590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:30.689851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:30.690500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:30.690806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:30.692197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:30.707095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 463/920] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:30.712853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:30.736106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:30.844129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39401746-7961-43fc-8f10-cce87c1e5a7a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:30.882144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:30.882703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:30.882981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:30.883284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:30.891423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:30.892882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:14:30.904137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:30.905207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:30.906426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 458/921] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:30.911707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:31.004558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:31.026245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa27-6225-41ae-9712-8906d873f696 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:31.062023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:31.063123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:31.063639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:31.063943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:31.065016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:31.065494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:31.067133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac96f38-c680-4964-82c8-f20edee25fa0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:31.084542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 464/922] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:14:31.092724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:31.176003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:31.204375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43977e5f-4428-4094-8c79-6179077fdc78 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:31.245139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:31.245809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:31.246085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:31.246354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/a8112d8c2ea24bd286a5f139ba765f2a/roles/256c302cf30b4fadbbe2ea07682646b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:31.264966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=a8112d8c2ea24bd286a5f139ba765f2a, role_id=256c302cf30b4fadbbe2ea07682646b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:31.266453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651652c-21c9-4e1c-8f19-e307e8315f9e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:31.281674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 459/923] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:14:31 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/a8112d8c2ea24bd286a5f139ba765f2a/roles/256c302cf30b4fadbbe2ea07682646b0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:14:33.451783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd5c5-5c4a-447e-821c-9ce18eecf78c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:33.452344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd5c5-5c4a-447e-821c-9ce18eecf78c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:33.452344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd5c5-5c4a-447e-821c-9ce18eecf78c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:33.453679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 465/924] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:33.459900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:33.460342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:33.460434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:33.479310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:33.480585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:33.549068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 460/925] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:33.813828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd5c5-5c4a-447e-821c-9ce18eecf78c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:33.915924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd5c5-5c4a-447e-821c-9ce18eecf78c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:33.950350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:33.951179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:33.951581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:33.952004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:33.960281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:33.964783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:14:33.969712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:33.970830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:33.976320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 466/926] 10.222.0.36 () {56 vars in 1102 bytes} [Tue Oct 25 13:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:33.977179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:34.036638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:34.082498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31877-1fce-44d4-bbac-14078c80d9b3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:34.121912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:34.122490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:34.122769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:34.123024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:34.123721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] RBAC: Adding query filter params (name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:34.124021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:34.125441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:34.133807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 461/927] 10.222.0.36 () {56 vars in 1103 bytes} [Tue Oct 25 13:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:34.139869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:34.254980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a1379-b286-4c1e-9e90-bdd7c1fba81c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:34.295684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:34.296343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:34.296600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:34.296874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:34.311445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:34.313350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:34.331682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:34.332873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:34.334113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:34.334113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:34.334113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:34.334113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 467/928] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:34.339679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:34.454770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ebd7-432d-45a8-ac32-055cdaa3f844 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:34.472605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:34.491696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:34.492481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:34.492801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:34.493096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:34.493828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:34.494137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:34.495571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:34.509550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 462/929] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:34.516261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:34.573310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:34.634872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131973-8ee9-474c-82d7-b32a6361b2e0 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:34.676645 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:34.677249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:34.677524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:34.677816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:34.686891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:34.688638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:14:34.703077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:34.704488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:34.705426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 468/930] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:34.711190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:34.823859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0e921-bd2f-4d86-8f1d-61b715937001 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:34.871468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:34.872074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:34.872347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:34.872608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:34.873335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:34.873631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:34.875184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:34.894302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 463/931] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Oct 25 13:14:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:34.900741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:35.005170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160fc3c-e6a3-4ab5-b7aa-dd835fc8b884 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:35.016431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:35.040226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:35.040831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:35.041102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:35.041401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:35.113985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:35.115488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:35.218556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 469/932] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:14:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:14:35.224963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:35.263621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:35.342672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d2b9a-9615-46c8-bdff-7a84969c9205 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:35.388486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:35.389218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:35.389494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:35.389802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:35.404496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:35.406157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:35.421965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:35.423199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:35.424562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:35.424562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:35.424562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:35.427801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 464/933] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:35.429857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:35.441833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:35.538712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ac791-7ae1-478f-9041-57c8c3ee24db None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:35.575294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:35.575978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:35.576326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:35.576660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:35.577463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:35.577854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:35.579276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:35.591127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 470/934] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:35.597149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:35.691131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:35.708350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a62fc-1827-4c70-b4a3-5940a7451e75 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:35.745233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:35.745897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:35.746191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:35.746453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:35.754048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:35.755531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:14:35.767320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:35.768383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:35.774508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 465/935] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:35.775073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:35.846463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:35.881313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd37-2089-4a68-a53a-fb3b5bced31f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:35.916419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:35.917042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:35.917323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:35.917648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:35.918356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:35.918660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:35.920138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0a802-228c-4abc-9ed4-3117dff84fa6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:35.932558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 471/936] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:35.939612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:36.051393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bba396-c4b6-4177-b340-0886cdbb9dae None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:36.088350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:36.088964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:36.089242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:36.089511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:36.090253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] RBAC: Adding query filter params (role.id=256c302cf30b4fadbbe2ea07682646b0, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=a8112d8c2ea24bd286a5f139ba765f2a) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:36.090564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:36.092045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194d985-e0d0-4075-a355-3a59bdd08540 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:36.106802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 466/937] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:14:35 2022] GET /identity/v3/role_assignments?user.id=a8112d8c2ea24bd286a5f139ba765f2a&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=256c302cf30b4fadbbe2ea07682646b0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:38.301811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2e1a-2621-4088-9ebc-b357fc0de55b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:38.302483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2e1a-2621-4088-9ebc-b357fc0de55b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:38.302483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2e1a-2621-4088-9ebc-b357fc0de55b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:38.303801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 472/938] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:14:38.309685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:38.310215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:38.310321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:38.330343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:38.332062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:38.334999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:38.410877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 467/939] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:38.695806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2e1a-2621-4088-9ebc-b357fc0de55b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:38.803103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2e1a-2621-4088-9ebc-b357fc0de55b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:38.832307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:38.838230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:38.838853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:38.839126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:38.839425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:38.849258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:14:38.854026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:38.855192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:38.856582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 473/940] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:38.862175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:38.965646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bb6d5-8049-4ae3-8609-d9fc6508af36 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.001668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.002346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.002651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.002995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.003795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:39.004121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.005479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.020220 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 468/941] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:39.021506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:39.130312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96856ea7-1f5e-408b-bb5d-465cd715ccef None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.168347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.168979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.169257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.169524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.183845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.185303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:39.200655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:39.201709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:39.202664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:39.202664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:39.202664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:39.202664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 474/942] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:39.208956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:39.212026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:39.318828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bc84-0891-48dd-94e9-e214c5033c5a None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.352175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.352785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.353063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.353328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.353970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:39.354262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.355666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.368858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 469/943] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:39.374650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:39.421376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:39.482526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6129-91eb-4b74-9d4b-198f902fb34e None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.519256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.519975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.520251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.520515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.528154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.529654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:14:39.541050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:39.542217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:39.543538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 475/944] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:39.548017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:39.613119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:39.669946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f70635-3b66-4b0a-a254-bc832cb85008 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.707856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.708565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.708771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.709082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.709870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:39.710214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.711903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.730091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 470/945] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:39.736372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:39.841883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1acc4d-1a0b-48fa-bd28-e9d1eacf6e5c None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:39.843520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:39.878266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:39.878793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:39.879067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:39.879365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:39.890458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:39.892050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:39.907279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:39.908424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:39.909819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:39.909819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:39.909819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:39.909819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 476/946] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:39.914503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:40.026030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077f975-36ed-4bc5-b391-dbf9d66ccc61 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:40.048756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:40.063477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:40.064099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:40.064427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:40.064701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:40.065354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:40.065683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:40.067056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:40.078759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 471/947] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:40.085242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:40.192901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35097f-59e8-49d3-a9d4-986d6e60af99 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:40.229876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:40.230509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:40.230781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:40.231092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:40.238089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:40.239554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:14:40.252039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:40.253198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:40.254103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 477/948] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:40.259720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:40.373508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515c08a-85a3-4952-a7a4-428017dc28dc None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:40.415808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:40.416446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:40.416713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:40.416990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:40.417689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:40.417987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:40.419466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b452e35-25d5-4cc0-b458-513e4cb1611b None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:40.433033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 472/949] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:40.439554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:40.458213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:40.558311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f521a0-0100-4282-bcaa-e649a9abf7b6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:40.597761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:40.598527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:40.598865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:40.599079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] PATH_INFO: `/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/a8112d8c2ea24bd286a5f139ba765f2a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:40.601015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb51413c1144daab7dc47dec8f28c10, user_id=a8112d8c2ea24bd286a5f139ba765f2a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:40.602445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec862fd8-1045-4ca4-a0ea-d60848c5cc2d None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:40.620957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 478/950] 10.222.0.36 () {58 vars in 1439 bytes} [Tue Oct 25 13:14:40 2022] PUT /identity/v3/projects/0bb51413c1144daab7dc47dec8f28c10/users/a8112d8c2ea24bd286a5f139ba765f2a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:14:42.782182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7902-e2e2-4b7e-a785-44724e8fb2fc None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:42.782789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7902-e2e2-4b7e-a785-44724e8fb2fc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:42.782789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7902-e2e2-4b7e-a785-44724e8fb2fc None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:42.784291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 473/951] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:42.790095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:42.790598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:42.790688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:42.809990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:42.811251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:42.889792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 479/952] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:43.183842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc7902-e2e2-4b7e-a785-44724e8fb2fc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:43.231333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:43.296465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc7902-e2e2-4b7e-a785-44724e8fb2fc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:43.332905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:43.333709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:43.334053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:43.334359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:43.345085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 25 13:14:43.350117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:43.351312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:43.352192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 474/953] 10.222.0.36 () {56 vars in 1096 bytes} [Tue Oct 25 13:14:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:43.358444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:43.362587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:43.466648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994e33-bbcd-469a-9abf-76a911dd2e27 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:43.503365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:43.504084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:43.504427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:43.504760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:43.505498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:43.505910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:43.507369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:43.519678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 480/954] 10.222.0.36 () {56 vars in 1099 bytes} [Tue Oct 25 13:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:43.525784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:43.636580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:43.639982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c7eb-14aa-4ea8-842b-9d524b04471e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:43.686986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:43.687839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:43.688247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:43.688667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:43.702360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:43.704906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:43.726981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:43.728189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:43.729103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:43.729103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:43.729103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:43.729103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 475/955] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:43.736203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:43.810829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:43.853856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f33ea-ab41-4b0b-8320-e4933639f01f None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:43.887074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:43.887717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:43.887995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:43.888264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:43.888951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:43.889254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:43.890664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:43.902387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 481/956] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:14:43.908582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:44.020714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfd4ca-9e2b-47a1-a054-23b8a6a31b59 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:44.061896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:44.062539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:44.062923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:44.063289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:44.073690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:44.075684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 25 13:14:44.091257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:44.092367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:44.093534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 476/957] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:44.099500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:44.204598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde0e6-9b50-470d-bad2-8b5ed4b15536 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:44.237917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:44.246936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:44.247595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:44.247863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:44.248130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:44.248862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:44.249202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:44.250630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:44.268951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 482/958] 10.222.0.36 () {56 vars in 1143 bytes} [Tue Oct 25 13:14:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:44.276110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:44.293008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:44.398500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c33536-1052-4075-9f26-2ade7a553968 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:44.435296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:44.435932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:44.436206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:44.436497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:44.506104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:44.508324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:44.524432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:44.625249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 477/959] 10.222.0.36 () {58 vars in 1282 bytes} [Tue Oct 25 13:14:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 25 13:14:44.632120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:44.750633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6b22e-89b3-4706-a660-5956d70249f6 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:44.791275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:44.791936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:44.792192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:44.792466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:44.802794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:44.804257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 25 13:14:44.821323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:44.823012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 25 13:14:44.823959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 25 13:14:44.823959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 25 13:14:44.823959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:44.823959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 483/960] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:44.829629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:44.950137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640a201-1346-44e7-86c4-18e81f812288 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:44.965214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:44.988381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:44.989027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:44.989247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:44.989538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:44.990195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:44.990490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:44.991934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:45.004821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 478/961] 10.222.0.36 () {56 vars in 1109 bytes} [Tue Oct 25 13:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:45.010798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:45.052964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:45.123741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6223fb8-8db4-4821-b077-603e53d52749 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:45.160092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:45.160727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:45.161002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:45.161250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:45.168760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:45.170247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 25 13:14:45.182690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:45.183866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: ERROR keystone.server.flask.application Oct 25 13:14:45.185063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 484/962] 10.222.0.36 () {56 vars in 1111 bytes} [Tue Oct 25 13:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:45.190251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:45.297994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5b416-5e21-489c-af73-90948b15139e None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:45.334650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:45.335262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:45.335532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:45.335833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:45.336537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:45.336835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:45.338273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b776d-5770-44df-8194-25afb29eecb4 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:45.351773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 479/963] 10.222.0.36 () {56 vars in 1148 bytes} [Tue Oct 25 13:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:45.358574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:45.463236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:45.468853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9ed84-21df-4eb2-8f29-9c4fe1294a38 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:45.501792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:45.502398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:45.502674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:45.502941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:45.503703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] RBAC: Adding query filter params (role.id=7ce3b8059d8449b1b4ebed7b09183e21, scope.project.id=0bb51413c1144daab7dc47dec8f28c10, user.id=a8112d8c2ea24bd286a5f139ba765f2a) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:45.504008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:45.505434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061e73b-573b-4078-9b96-d7cf6a1d64b2 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:45.511377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 485/964] 10.222.0.36 () {56 vars in 1374 bytes} [Tue Oct 25 13:14:45 2022] GET /identity/v3/role_assignments?user.id=a8112d8c2ea24bd286a5f139ba765f2a&scope.project.id=0bb51413c1144daab7dc47dec8f28c10&role.id=7ce3b8059d8449b1b4ebed7b09183e21 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:14:47.662137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2985228-7e97-43f9-922c-bb7afb63669b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:47.662601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2985228-7e97-43f9-922c-bb7afb63669b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:47.662689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2985228-7e97-43f9-922c-bb7afb63669b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:47.664357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 480/965] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:47.670685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:47.671404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:47.671581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:47.696346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:47.697541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:47.774165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 486/966] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:48.040721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2985228-7e97-43f9-922c-bb7afb63669b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:48.077452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:48.156222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2985228-7e97-43f9-922c-bb7afb63669b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:48.192990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:48.193613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:48.193885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:48.194149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:48.194736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:48.196136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:48.205139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:48.206656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:48.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 481/967] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:14:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:48.216365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:48.313694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:48.324663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a690d2-9bcb-40e8-b76b-fb8faa82b723 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:48.366401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:48.367412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:48.367890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:48.368282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:48.369175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:48.369568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:48.371948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbf67-5c00-4c7c-9c8b-7d64a3bdb8b1 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:48.377322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 487/968] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Oct 25 13:14:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:48.384597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:48.499919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:48.509030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965553e-9f7b-4ccb-af63-eb3fbce66a81 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:48.553411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:48.554013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:48.554289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:48.554557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] PATH_INFO: `/v3/services` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:48.555192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:48.555523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:48.556896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664cc4c-9084-486a-8981-1a5f0af203eb None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:48.561495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 482/969] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Oct 25 13:14:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:50.687049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6bf2-8777-44f7-8654-ad621b4a0edb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:50.687981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6bf2-8777-44f7-8654-ad621b4a0edb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:50.688122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da6bf2-8777-44f7-8654-ad621b4a0edb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:50.690182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 488/970] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:50.698130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1a81-d9f8-4caf-97a7-52ac05ad9ace None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:50.698580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1a81-d9f8-4caf-97a7-52ac05ad9ace None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:50.698671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1a81-d9f8-4caf-97a7-52ac05ad9ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:50.719124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8e1c1a81-d9f8-4caf-97a7-52ac05ad9ace None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:50.720334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8e1c1a81-d9f8-4caf-97a7-52ac05ad9ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:50.799329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 483/971] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:14:51.063566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da6bf2-8777-44f7-8654-ad621b4a0edb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:51.170123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da6bf2-8777-44f7-8654-ad621b4a0edb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:51.205819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:51.206774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:51.207050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:51.207349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:51.208013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:51.209375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8131d-6661-4b4e-b4c1-742536bd74dd None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:51.227795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 489/972] 10.222.0.36 () {60 vars in 1141 bytes} [Tue Oct 25 13:14:51 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:14:53.370535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115bcd-f833-4b8e-8c2b-816fa31ab807 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:53.371082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115bcd-f833-4b8e-8c2b-816fa31ab807 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:53.371082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115bcd-f833-4b8e-8c2b-816fa31ab807 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:53.372663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 484/973] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:53.378491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:53.378967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:53.379073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:53.398955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:53.400163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:53.410642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:53.477919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 490/974] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 25 13:14:53.747379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19115bcd-f833-4b8e-8c2b-816fa31ab807 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:53.847176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:53.858765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19115bcd-f833-4b8e-8c2b-816fa31ab807 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:53.893308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:53.893869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:53.894144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:53.894415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:53.895003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:53.896410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:53.905426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:53.906542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:53.907726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:53.911515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 485/975] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:14:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:53.913045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:54.029376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4131c2-9033-40fb-b798-43687e0ee39e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:54.068511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:54.069120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:54.069396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:54.069671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:54.070310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:54.070611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:54.072016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:54.077861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 491/976] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Oct 25 13:14:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:54.084705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:54.187581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd058d-1b57-441a-a315-342904a763fd None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:54.222298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:54.222889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:54.223196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:54.223463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:54.224164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] RBAC: Adding query filter params (interface=public, service_id=cf8e68d6c0fb415db3644e1a96af340d, region_id=RegionOne) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:54.224473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:54.225839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d95f81-5505-426f-9410-7b9bbe534b9e None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:54.230794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 486/977] 10.222.0.36 () {56 vars in 1251 bytes} [Tue Oct 25 13:14:54 2022] GET /identity/v3/endpoints?service_id=cf8e68d6c0fb415db3644e1a96af340d&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:54.236722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:54.241186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:54.341481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964896c-c504-42a7-9f57-68becd0335d4 None admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:54.382299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:54.382919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:54.383178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:54.383439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:54.384040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:54.385405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b94e3-963e-4146-a65d-b9c2d6f25937 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:54.392305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 492/978] 10.222.0.36 () {56 vars in 1087 bytes} [Tue Oct 25 13:14:54 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:14:56.502065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727495-a36f-42be-af3f-2333d0404e7d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:56.502908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727495-a36f-42be-af3f-2333d0404e7d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:56.502908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727495-a36f-42be-af3f-2333d0404e7d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:56.505090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 487/979] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:14:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:14:56.513191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:56.513620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:56.513709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:56.533482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:14:56.534654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:14:56.599823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:56.606232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 493/980] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:14:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 25 13:14:56.871049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727495-a36f-42be-af3f-2333d0404e7d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:56.896698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:56.994555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727495-a36f-42be-af3f-2333d0404e7d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:57.030142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:57.030760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:57.031032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:57.031382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:57.031949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:57.033313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 25 13:14:57.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 25 13:14:57.043365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.get_or_create( Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application with Lock( Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self._enter() Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application return self.creator() Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application created_value = creator( Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: ERROR keystone.server.flask.application Oct 25 13:14:57.044574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 488/981] 10.222.0.36 () {56 vars in 1117 bytes} [Tue Oct 25 13:14:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 25 13:14:57.050131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:57.156954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96adf6d-9bd6-4099-a171-650e4d2fadcd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:57.194342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:57.195370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:57.195837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:57.196284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] PATH_INFO: `/v3/services` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:57.197324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:14:57.197822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:57.200088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c626d-68e0-4028-943b-298ebf40b554 None admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:57.207361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 494/982] 10.222.0.36 () {56 vars in 1116 bytes} [Tue Oct 25 13:14:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:14:57.213389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:14:57.300826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:14:57.321653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c56a84-2275-4034-9a3c-1bc843311b40 None admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:14:57.359075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:14:57.359741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:14:57.360112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:14:57.360432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:14:57.361085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:14:57.362537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d18f6b-b17e-4efe-86ad-8fa44ef3b3d9 None admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:14:57.399184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 489/983] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:14:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:15:07.203383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=69 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:15:07.379383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:00.613390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48128b-2fb6-4c2d-a37f-173fa8155948 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:00.613920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48128b-2fb6-4c2d-a37f-173fa8155948 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:00.613920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48128b-2fb6-4c2d-a37f-173fa8155948 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:00.615385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 495/984] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:00.620907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0c97-392f-4c8a-9420-cbe02978e07e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:00.622033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0c97-392f-4c8a-9420-cbe02978e07e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:00.622477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0c97-392f-4c8a-9420-cbe02978e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:00.793505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-59cd0c97-392f-4c8a-9420-cbe02978e07e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:00.795363 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59cd0c97-392f-4c8a-9420-cbe02978e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:01.014740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 490/985] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:16:01.317028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:01.317313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:01.317583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:01.319689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 496/986] 10.222.0.36 () {54 vars in 964 bytes} [Tue Oct 25 13:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:01.325957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384aa-36ad-4485-8dae-b6e6f93f2b4c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:01.326326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384aa-36ad-4485-8dae-b6e6f93f2b4c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:01.326517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d384aa-36ad-4485-8dae-b6e6f93f2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:01.432919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-78d384aa-36ad-4485-8dae-b6e6f93f2b4c None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:01.434269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-78d384aa-36ad-4485-8dae-b6e6f93f2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:01.459342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:01.521716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 491/987] 10.222.0.36 () {58 vars in 1061 bytes} [Tue Oct 25 13:16:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:01.531178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:01.533343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:01.651921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d424c17-fab8-4229-9452-f0be2a5ae81c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:01.740350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:01.740885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:01.741161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:01.741422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:01.743200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:01.806115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:01.878541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:01.880071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e172b79-1ba6-4bd8-bc2f-1bdd4770add7 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:01.903988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 497/988] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:04.370047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:04.370649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:04.370649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:04.372716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 492/989] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:04.381120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca702d-e4a0-4e1f-bbb0-2c53df4394e7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:04.381647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca702d-e4a0-4e1f-bbb0-2c53df4394e7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:04.381736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca702d-e4a0-4e1f-bbb0-2c53df4394e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:04.462138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5bca702d-e4a0-4e1f-bbb0-2c53df4394e7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:04.463385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bca702d-e4a0-4e1f-bbb0-2c53df4394e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:04.562223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 498/990] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:04.843435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:04.844983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:04.953351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31371548-b7db-4575-ac16-b917f9a35545 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:05.028048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:05.030436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:05.031202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:05.031536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:05.032098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:05.034332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:05.155958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:05.157417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a7b68-33a8-4f97-8b66-3a421c9dab1a service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:05.175862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 493/991] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:16:07.640061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19056b-edc8-4bd5-bbe4-d0eb0206bead None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:07.640768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19056b-edc8-4bd5-bbe4-d0eb0206bead None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:07.640910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19056b-edc8-4bd5-bbe4-d0eb0206bead None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:07.643072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 499/992] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:07.652157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d0dd1-01f0-4376-9a14-db2ddded47b7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:07.652571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d0dd1-01f0-4376-9a14-db2ddded47b7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:07.652701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d0dd1-01f0-4376-9a14-db2ddded47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:07.725668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e07d0dd1-01f0-4376-9a14-db2ddded47b7 None None] MFA Rules not processed for user `ad176173d8984450b1076196825032c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:07.726892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e07d0dd1-01f0-4376-9a14-db2ddded47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:07.780635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:07.827072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 494/993] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:08.122752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:08.123028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:08.123336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:08.124970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 500/994] 10.222.0.36 () {54 vars in 964 bytes} [Tue Oct 25 13:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:08.130621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a2d-dace-40e5-9c0f-52abb405c3b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:08.130909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a2d-dace-40e5-9c0f-52abb405c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:08.131238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24645a2d-dace-40e5-9c0f-52abb405c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:08.213437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24645a2d-dace-40e5-9c0f-52abb405c3b5 None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:08.214648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24645a2d-dace-40e5-9c0f-52abb405c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:08.290518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 495/995] 10.222.0.36 () {58 vars in 1061 bytes} [Tue Oct 25 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:08.299724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:08.301271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:08.311380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:08.409673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8c335-ee20-465b-ab6d-18521a51153b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:08.491298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:08.491911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:08.492193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:08.492466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:08.494074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:08.560640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:08.609216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:08.610642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc17d34-1986-4cd8-a557-d02957a3b2a6 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:08.637601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 501/996] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:11.683561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:11.684137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:11.684137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:11.685462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 496/997] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:11.691881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9978228-5a00-408d-9b0d-6ef1f20dca4b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:11.692675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9978228-5a00-408d-9b0d-6ef1f20dca4b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:11.692807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9978228-5a00-408d-9b0d-6ef1f20dca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:11.765884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b9978228-5a00-408d-9b0d-6ef1f20dca4b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:11.767073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b9978228-5a00-408d-9b0d-6ef1f20dca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:11.869091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 502/998] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:12.151422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:12.153015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:12.215209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:12.279281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c37d63-d1b5-4e01-8b9b-260026e88e5c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:12.354047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:12.354653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:12.354923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:12.355179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:12.357071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:12.467275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:12.487806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:12.489267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ab90f-a141-4757-9178-29b36a1761ae service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:12.509740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 497/999] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:16:15.313917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fc2a1-645f-401d-af35-0a826612273c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:15.314497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fc2a1-645f-401d-af35-0a826612273c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:15.314497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fc2a1-645f-401d-af35-0a826612273c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:15.315876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 503/1000] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:15.321320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35036-b49a-49a3-9c7f-f77d5a51f0ca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:15.321748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35036-b49a-49a3-9c7f-f77d5a51f0ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:15.321852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35036-b49a-49a3-9c7f-f77d5a51f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:15.401969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8fd35036-b49a-49a3-9c7f-f77d5a51f0ca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:15.403215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8fd35036-b49a-49a3-9c7f-f77d5a51f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:15.513553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 498/1001] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:15.797458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fc2a1-645f-401d-af35-0a826612273c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:15.799103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-431fc2a1-645f-401d-af35-0a826612273c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:15.820173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:15.905299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fc2a1-645f-401d-af35-0a826612273c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:15.981224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:15.981823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:15.982098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:15.982368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:15.983987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:16.058380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:16.098563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:16.100032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06221969-3d99-41d7-8129-b0cbd1f8895a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:16.118619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 504/1002] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:18.304561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:18.305119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:18.305119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:18.306466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 499/1003] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:18.311826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82701-466e-42f8-b31b-8f8669c422e5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:18.312238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82701-466e-42f8-b31b-8f8669c422e5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:18.312381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82701-466e-42f8-b31b-8f8669c422e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:18.382124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ffd82701-466e-42f8-b31b-8f8669c422e5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:18.383491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ffd82701-466e-42f8-b31b-8f8669c422e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:18.486853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 505/1004] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:18.762931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:18.764563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:18.825344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:18.869144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9d72-46fd-4b46-ba4b-9d44608d8492 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:18.946788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:18.947339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:18.947615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:18.947885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:18.949475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:19.067113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:19.069584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:19.071060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b75ff-00b3-405b-a472-fbc7745d7b76 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:19.090236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 500/1005] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:21.293859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:21.294397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:21.294397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:21.295715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 506/1006] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:21.301778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb84188-30e8-484a-b9a9-0363efdd6a79 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:21.302216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb84188-30e8-484a-b9a9-0363efdd6a79 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:21.302411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb84188-30e8-484a-b9a9-0363efdd6a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:21.370625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fdb84188-30e8-484a-b9a9-0363efdd6a79 None None] MFA Rules not processed for user `ad176173d8984450b1076196825032c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:21.371873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fdb84188-30e8-484a-b9a9-0363efdd6a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:21.474374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 501/1007] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:21.751995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:21.753686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:21.785518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:21.861445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7a9bf-ad86-43b7-ab82-96d6d815b2f3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:21.937205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:21.937816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:21.938091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:21.938354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:21.939986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:22.020104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:22.053439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:22.054879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e079-8396-41d4-b125-1f03d7d7220b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:22.074143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 507/1008] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:16:24.463801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:24.464415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:24.464415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:24.465735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 502/1009] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:24.472116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f1811-f160-4f84-94a4-ad35c5c767a0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:24.472560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f1811-f160-4f84-94a4-ad35c5c767a0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:24.472651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f1811-f160-4f84-94a4-ad35c5c767a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:24.551382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-347f1811-f160-4f84-94a4-ad35c5c767a0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:24.552589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-347f1811-f160-4f84-94a4-ad35c5c767a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:24.658350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 508/1010] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:24.937869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:24.939445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:25.011846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:25.049476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e84e-4769-4dd9-8d3e-44472489fe0d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:25.129187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:25.129802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:25.130084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:25.130352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:25.131975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:25.265023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:25.266466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89ae68-b325-4f14-8b7e-ef7721c69df0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:25.277294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:25.286840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 503/1011] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:16:27.476352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:27.476930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:27.476930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:27.478137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 509/1012] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:27.484671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b97f5-e0ca-41a3-846f-924b1ca9dff0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:27.485373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b97f5-e0ca-41a3-846f-924b1ca9dff0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:27.485560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b97f5-e0ca-41a3-846f-924b1ca9dff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:27.565897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4d3b97f5-e0ca-41a3-846f-924b1ca9dff0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:27.567102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d3b97f5-e0ca-41a3-846f-924b1ca9dff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:27.673432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 504/1013] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:27.977105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:27.979601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:28.030589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:28.105083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453439f2-fa2d-41e5-bff1-9203d681d223 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:28.185662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:28.186266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:28.186539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:28.186801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:28.188443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:28.273586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:28.301783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:28.303260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef442107-44e0-4dd1-b74b-3bb9fe45b8df service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:28.322162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 510/1014] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:30.935860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:30.936618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:30.936618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:30.938578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 505/1015] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:30.944824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e56d-7686-465b-be69-163fc80d8c72 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:30.945226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e56d-7686-465b-be69-163fc80d8c72 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:30.945398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e56d-7686-465b-be69-163fc80d8c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:31.012313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5d17e56d-7686-465b-be69-163fc80d8c72 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:31.013510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5d17e56d-7686-465b-be69-163fc80d8c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:31.112886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 511/1016] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:31.388935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:31.390520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:31.473939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:31.508149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb7e41-882e-402e-9329-e3c01ecea0c2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:31.584057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:31.584668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:31.584943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:31.585212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:31.586814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:31.705320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:31.706801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b70e5-0b00-4939-a00a-a7eb27f6866b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:31.725464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 506/1017] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:35.902640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:35.903192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:35.903192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:35.904509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 512/1018] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:35.909885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae0e-55f2-4da0-8166-711a152e2f6d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:35.910304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae0e-55f2-4da0-8166-711a152e2f6d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:35.910390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae0e-55f2-4da0-8166-711a152e2f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:35.916180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:35.978608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-843bae0e-55f2-4da0-8166-711a152e2f6d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:35.979767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-843bae0e-55f2-4da0-8166-711a152e2f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:36.077562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 507/1019] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:36.354254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:36.355901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:36.406167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:36.463625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264005a-942b-4939-a115-cb3da45e20b9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:36.539491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:36.540102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:36.540382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:36.540654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:36.542271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:36.643676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:36.660727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:36.662189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db25698-21a6-465a-9299-5bd412c704c5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:36.680292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 513/1020] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:39.434121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:39.434691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:39.434691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:39.436069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 508/1021] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:39.441373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f36f8-d9d5-460d-b112-881f29746729 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:39.441791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f36f8-d9d5-460d-b112-881f29746729 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:39.441880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f36f8-d9d5-460d-b112-881f29746729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:39.513475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3c2f36f8-d9d5-460d-b112-881f29746729 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:39.514672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c2f36f8-d9d5-460d-b112-881f29746729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:39.618226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 514/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:39.895403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:39.896963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:39.994213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:40.011854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88ea0-aa86-4591-aecb-f45c69e8fcac None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:40.089935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:40.090553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:40.090839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:40.091098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:40.092737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:40.206973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:40.208842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7016e9-3a13-4ca7-8409-49f25c6ea877 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:40.228226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 509/1023] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:44.313048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:44.313536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:44.313710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:44.315005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 515/1024] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:44.321068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c037-8033-4abe-811b-e71fcecf803c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:44.321504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c037-8033-4abe-811b-e71fcecf803c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:44.321626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c037-8033-4abe-811b-e71fcecf803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:44.338742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:44.394300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5324c037-8033-4abe-811b-e71fcecf803c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:44.395555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5324c037-8033-4abe-811b-e71fcecf803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:44.496849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 510/1025] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:44.777689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:16:44.779273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:44.838118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:44.885657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5df25f-697e-4a3d-a4fb-da304ee2e9a0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:16:44.966511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:16:44.967178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:44.967459 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:44.967726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:44.969328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:45.086604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:45.090940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:16:45.092525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b671-dcb4-44f1-abb5-e6e72fb16e86 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:16:45.110849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 516/1026] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:16:50.046584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38817d10-1083-4d11-b87d-6e72686c86e7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:50.047491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38817d10-1083-4d11-b87d-6e72686c86e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:50.047925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38817d10-1083-4d11-b87d-6e72686c86e7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:50.049698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 511/1027] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:50.055017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367021-d636-4284-b5d4-ae07a081c1fe None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:50.055431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367021-d636-4284-b5d4-ae07a081c1fe None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:50.055600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367021-d636-4284-b5d4-ae07a081c1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:50.151556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1a367021-d636-4284-b5d4-ae07a081c1fe None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:50.152750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a367021-d636-4284-b5d4-ae07a081c1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:50.287467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 517/1028] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:16:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:54.362932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f37f-3c89-4c01-acbd-020dcbf94bcf None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:54.363512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f37f-3c89-4c01-acbd-020dcbf94bcf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:54.363834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f37f-3c89-4c01-acbd-020dcbf94bcf None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:54.365265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 512/1029] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:54.370637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cc92-c8a1-49d0-bfd1-df163f5a6b12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:54.371057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cc92-c8a1-49d0-bfd1-df163f5a6b12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:54.371255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cc92-c8a1-49d0-bfd1-df163f5a6b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:54.428719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:54.455127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9904cc92-c8a1-49d0-bfd1-df163f5a6b12 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:54.456800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9904cc92-c8a1-49d0-bfd1-df163f5a6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:54.495524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=59 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:54.572770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 518/1030] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:16:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:16:55.963586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded60-c4c8-4332-87c9-999ec85a3ec5 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:55.964097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded60-c4c8-4332-87c9-999ec85a3ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:55.964097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded60-c4c8-4332-87c9-999ec85a3ec5 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:55.965385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 513/1031] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:55.970373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56547-8540-455b-9f40-eb542f44cf79 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:55.970822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56547-8540-455b-9f40-eb542f44cf79 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:55.970919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f56547-8540-455b-9f40-eb542f44cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:56.049604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99f56547-8540-455b-9f40-eb542f44cf79 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:56.050854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99f56547-8540-455b-9f40-eb542f44cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:56.095524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:16:56.163099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 519/1032] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:16:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:16:59.617761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9a09f-bb35-4873-a801-db75325781f6 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:59.618275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9a09f-bb35-4873-a801-db75325781f6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:59.618579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9a09f-bb35-4873-a801-db75325781f6 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:59.621211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 514/1033] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:16:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:16:59.626966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8da26-197c-47a0-80e4-1cab633db799 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:16:59.628108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8da26-197c-47a0-80e4-1cab633db799 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:16:59.628609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8da26-197c-47a0-80e4-1cab633db799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:16:59.699602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e0c8da26-197c-47a0-80e4-1cab633db799 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:16:59.701044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e0c8da26-197c-47a0-80e4-1cab633db799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:16:59.813834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 520/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:00.111979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:00.112638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:00.113113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:00.114793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 515/1035] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:00.130090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ebe7-3d7c-4bf0-b2c7-32fbf174cc6f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:00.130755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ebe7-3d7c-4bf0-b2c7-32fbf174cc6f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:00.131416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ebe7-3d7c-4bf0-b2c7-32fbf174cc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:00.156773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:00.226010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-72b8ebe7-3d7c-4bf0-b2c7-32fbf174cc6f None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:00.227441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-72b8ebe7-3d7c-4bf0-b2c7-32fbf174cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:00.370693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 521/1036] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:00.380025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:00.381548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:00.536342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc333f-abe8-4ce9-9749-18e669439e00 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:00.619761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:00.663048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:00.664007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:00.664598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:00.664875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:00.666712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:00.769440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:00.770903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409d0ee-7ec9-4ae5-bab5-eefa5799262b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:00.790880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 516/1037] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:03.141188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:03.141840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:03.141840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:03.143452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 522/1038] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:03.148418 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e447d-3464-4c4c-9490-d33bef3068a8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:03.148929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e447d-3464-4c4c-9490-d33bef3068a8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:03.149166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e447d-3464-4c4c-9490-d33bef3068a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:03.217557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bc6e447d-3464-4c4c-9490-d33bef3068a8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:03.218794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc6e447d-3464-4c4c-9490-d33bef3068a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:03.258265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:03.320944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 517/1039] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:03.604877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:03.606404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:03.693494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:03.788917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb27a9-3f75-4fa6-9bf8-5c648434e1ab None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:03.908552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:03.909206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:03.909483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:03.909749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:03.911425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:03.934681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:04.003632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:04.005229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c48069-7f14-4293-8a0a-687a57ecbd87 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:04.024506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 523/1040] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:06.507364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2454-bd12-4dfa-a770-17250d0184c3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:06.507929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2454-bd12-4dfa-a770-17250d0184c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:06.508306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a2454-bd12-4dfa-a770-17250d0184c3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:06.510167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 518/1041] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:06.515597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306811f3-023f-4664-bd1e-463474b86b6a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:06.516021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306811f3-023f-4664-bd1e-463474b86b6a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:06.516120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306811f3-023f-4664-bd1e-463474b86b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:06.581744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-306811f3-023f-4664-bd1e-463474b86b6a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:06.582960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-306811f3-023f-4664-bd1e-463474b86b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:06.684656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:06.694122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 524/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:06.972669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:06.972954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:06.973226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:06.974683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 519/1043] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:06.980017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893f46-74d2-438b-b1ac-ba0fa32377a6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:06.980402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893f46-74d2-438b-b1ac-ba0fa32377a6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:06.980797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893f46-74d2-438b-b1ac-ba0fa32377a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:07.062503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3893f46-74d2-438b-b1ac-ba0fa32377a6 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:07.063881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3893f46-74d2-438b-b1ac-ba0fa32377a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:07.184760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 525/1044] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:07.192903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:07.194647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:07.347403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cb82e-6acc-485c-a77d-9d60965b7d5a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:07.368324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:07.475689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:07.476305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:07.476575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:07.476825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:07.478421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:07.580445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:07.582247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec77c5-9c00-4c4b-bd7e-797b486aeea8 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:07.602079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 520/1045] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:07.658685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacea55-32dd-41e3-bdc1-55039c6c189c None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:07.659324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacea55-32dd-41e3-bdc1-55039c6c189c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:07.659781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacea55-32dd-41e3-bdc1-55039c6c189c None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:07.661556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 526/1046] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:07.667078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a116-9991-4af5-a6e9-679ea121163c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:07.667468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a116-9991-4af5-a6e9-679ea121163c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:07.667849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a116-9991-4af5-a6e9-679ea121163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:07.708534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:07.757317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2615a116-9991-4af5-a6e9-679ea121163c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:07.758750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2615a116-9991-4af5-a6e9-679ea121163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:07.876137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 521/1047] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:08.205756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:08.205756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:08.205756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:08.206933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 527/1048] 10.222.0.36 () {54 vars in 963 bytes} [Tue Oct 25 13:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:08.212673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5974d0-29aa-4ea4-9ca8-0e3f1cf1724e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:08.213037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5974d0-29aa-4ea4-9ca8-0e3f1cf1724e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:08.213380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5974d0-29aa-4ea4-9ca8-0e3f1cf1724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:08.296943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bb5974d0-29aa-4ea4-9ca8-0e3f1cf1724e None None] MFA Rules not processed for user `7623623b296e4d04949973891b697c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:08.298170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bb5974d0-29aa-4ea4-9ca8-0e3f1cf1724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:08.359840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:08.420135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 522/1049] 10.222.0.36 () {58 vars in 1060 bytes} [Tue Oct 25 13:17:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:08.420363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:08.421599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:08.504473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:08.581983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960e5dc-b93f-4eaa-b2e4-a5195abe7e8e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:08.695744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:08.696395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:08.696641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:08.696912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:08.698578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:08.843893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:08.854128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:08.855769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be9f95-e260-4763-b032-2043f51fee4e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:08.875655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 528/1050] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:09.832781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:09.833315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:09.833315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:09.835129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 523/1051] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:09.840478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7784-a42d-4aea-92ab-0e727232f2b7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:09.840909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7784-a42d-4aea-92ab-0e727232f2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:09.841002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7784-a42d-4aea-92ab-0e727232f2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:09.909461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-805d7784-a42d-4aea-92ab-0e727232f2b7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:09.910670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-805d7784-a42d-4aea-92ab-0e727232f2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:10.009743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 529/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:10.304086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:10.305599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:10.472022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad847c-9856-43eb-9230-707f3e443280 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:10.528813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:10.591545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:10.592147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:10.592411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:10.592684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:10.594260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:10.690361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:10.692045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b9fd3-6d16-4063-8a41-838cd621abef service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:10.710807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 524/1053] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:12.906578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:12.907125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:12.907125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:12.908578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 530/1054] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:12.914059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8399a9-f938-4f45-8b35-9f86ec4d84be None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:12.914509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8399a9-f938-4f45-8b35-9f86ec4d84be None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:12.914600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8399a9-f938-4f45-8b35-9f86ec4d84be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:12.986344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3e8399a9-f938-4f45-8b35-9f86ec4d84be None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:12.988414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e8399a9-f938-4f45-8b35-9f86ec4d84be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:12.997296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:13.094119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 525/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:13.383460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:13.385294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:13.454284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:13.544934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb0b67-4622-4c4b-8ff6-cd7dc093a0b3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:13.668088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:13.668695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:13.669073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:13.669350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:13.670922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:13.690280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:13.766665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:13.768334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8dd5-1f04-4efa-85c3-080ffebdb671 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:13.787540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 531/1056] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:15.985588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:15.986121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:15.986121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:15.991201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 526/1057] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:15.992665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a707d9-6e5a-4cad-8b9f-2001b65a43f0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:15.993080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a707d9-6e5a-4cad-8b9f-2001b65a43f0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:15.993203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a707d9-6e5a-4cad-8b9f-2001b65a43f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:16.066507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-06a707d9-6e5a-4cad-8b9f-2001b65a43f0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:16.067930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-06a707d9-6e5a-4cad-8b9f-2001b65a43f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:16.214039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:16.224318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 532/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:16.513491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:16.515295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:16.670007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:16.685145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8fedd-fc36-4fae-a85e-f45e1d58f0ba None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:16.826756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:16.827385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:16.827672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:16.827955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:16.829698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:16.937758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:16.939444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34de9a-5fe1-4c76-a8fd-db635939a991 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:16.949590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:16.958944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 527/1059] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:19.189081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:19.189627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:19.189627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:19.191208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 533/1060] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:19.197529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90fcf-ca2d-4588-92a2-3af2771fdea9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:19.198019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90fcf-ca2d-4588-92a2-3af2771fdea9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:19.198218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90fcf-ca2d-4588-92a2-3af2771fdea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:19.269806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fcd90fcf-ca2d-4588-92a2-3af2771fdea9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:19.271108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fcd90fcf-ca2d-4588-92a2-3af2771fdea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:19.371985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 528/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:19.655429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:19.657133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:19.841221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef82a03-27cc-44a0-a690-fea7927da388 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:19.915329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:19.962444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:19.963223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:19.963579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:19.963854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:19.965532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:20.058899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:20.060448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a1c3-bafe-420c-9611-23aefd5fd2f3 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:20.078716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 534/1062] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:17:22.325557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796393-f86f-4e3b-8850-7761342011cf None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:22.326158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796393-f86f-4e3b-8850-7761342011cf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:22.326158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796393-f86f-4e3b-8850-7761342011cf None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:22.327530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 529/1063] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:22.332906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47574d7-b8f9-42df-adf7-4c7ec0b29722 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:22.333330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47574d7-b8f9-42df-adf7-4c7ec0b29722 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:22.333455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47574d7-b8f9-42df-adf7-4c7ec0b29722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:22.403194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a47574d7-b8f9-42df-adf7-4c7ec0b29722 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:22.404673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a47574d7-b8f9-42df-adf7-4c7ec0b29722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:22.438273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:22.505669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 535/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:22.790986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05796393-f86f-4e3b-8850-7761342011cf None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:22.792726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05796393-f86f-4e3b-8850-7761342011cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:22.874227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:22.944676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05796393-f86f-4e3b-8850-7761342011cf None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:23.058312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:23.058932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:23.059229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:23.059499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:23.061091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:23.103224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:23.156711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:23.158362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8538cb-546a-4a98-aa44-03b8b66fb730 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:23.177016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 530/1065] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:23.968524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3475d-2859-43ee-aded-aec65517ab41 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:23.969087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3475d-2859-43ee-aded-aec65517ab41 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:23.969087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3475d-2859-43ee-aded-aec65517ab41 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:23.970353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 536/1066] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:23.975923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b50c83-b3d3-4c3e-8b6a-8c445a524315 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:23.976351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b50c83-b3d3-4c3e-8b6a-8c445a524315 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:23.976443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b50c83-b3d3-4c3e-8b6a-8c445a524315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:24.045859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c5b50c83-b3d3-4c3e-8b6a-8c445a524315 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:24.047081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5b50c83-b3d3-4c3e-8b6a-8c445a524315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:24.149081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 531/1067] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:24.449103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:24.449377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:24.449642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:24.451028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 537/1068] 10.222.0.36 () {54 vars in 962 bytes} [Tue Oct 25 13:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:24.456603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eefb2-50a1-48e8-bf06-79944d425f37 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:24.456958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eefb2-50a1-48e8-bf06-79944d425f37 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:24.457296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eefb2-50a1-48e8-bf06-79944d425f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:24.473873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:24.536283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8d9eefb2-50a1-48e8-bf06-79944d425f37 None None] MFA Rules not processed for user `7e49839dc5bc47d6a9a91800fef249c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:24.537495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d9eefb2-50a1-48e8-bf06-79944d425f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:24.616635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 532/1069] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Oct 25 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:24.627450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:24.629085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:24.736281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbedae-7fe9-4396-9d86-6401d2a890d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:24.798930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:24.814450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:24.815062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:24.815397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:24.815701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:24.817288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:24.929291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:24.930755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ef926-c7be-4c52-84c7-a12c8c26c872 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:24.949630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 538/1070] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:17:25.343960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d321-95fc-4260-aa40-6ca9f24b7f06 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:25.344421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d321-95fc-4260-aa40-6ca9f24b7f06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:25.344624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d321-95fc-4260-aa40-6ca9f24b7f06 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:25.345884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 533/1071] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:25.351788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:25.352270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:25.352270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:25.425750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:25.426950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:25.474149 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:25.561538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 539/1072] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:25.723612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652638-eb7a-4213-9c62-11a27c6449f3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:25.724044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652638-eb7a-4213-9c62-11a27c6449f3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:25.724520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652638-eb7a-4213-9c62-11a27c6449f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:25.797453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-0b652638-eb7a-4213-9c62-11a27c6449f3 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:25.798530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0b652638-eb7a-4213-9c62-11a27c6449f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:25.850100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:25.851356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:25.861619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:25.896274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 534/1073] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:25.943468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a19b-afeb-44ed-8d02-d06ca7f7addd None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:25.943758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a19b-afeb-44ed-8d02-d06ca7f7addd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:25.944032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a19b-afeb-44ed-8d02-d06ca7f7addd None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:25.945547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 535/1074] 10.222.0.36 () {54 vars in 943 bytes} [Tue Oct 25 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:17:25.951229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:25.951476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:25.951753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:25.976615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286b28-5ba6-4c10-bbe2-5473d945891f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:26.030181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] MFA Rules not processed for user `084038e0639b49c99485f53463392d11`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:26.031462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:26.032799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:26.094474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:26.095449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:26.095728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:26.096001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:26.097620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:26.138044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 536/1075] 10.222.0.36 () {58 vars in 1040 bytes} [Tue Oct 25 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:26.147799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:26.157767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe89d95-d0f1-4c46-8929-d75c20d2c27d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:26.184685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:26.189693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:26.191405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c9688-9d59-4ba5-b9ca-080ed82204ca service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:26.210550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 540/1076] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:26.268486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:26.269118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:26.269398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:26.269668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:26.271424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:26.368551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:26.370210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02559c-9c79-4617-81b7-47186ff9e4e9 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:26.380306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 537/1077] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 25 13:17:28.442840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:28.443440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:28.443524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:28.445352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 541/1078] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:28.450544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17300f8-96b1-4b6a-9e46-0d7f31a64c01 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:28.451430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17300f8-96b1-4b6a-9e46-0d7f31a64c01 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:28.451712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17300f8-96b1-4b6a-9e46-0d7f31a64c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:28.531138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f17300f8-96b1-4b6a-9e46-0d7f31a64c01 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:28.532396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f17300f8-96b1-4b6a-9e46-0d7f31a64c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:28.540500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:28.656110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 538/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:17:28.932089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:28.933613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:29.005872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:29.097076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d71c47-2586-4d9e-8f4e-695e3aa69322 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:29.219530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:29.220264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:29.220529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:29.220794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:29.222495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:29.253240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:29.322520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:29.324762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f6787-7518-42f7-9faf-f2880901866f service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:29.349543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 542/1080] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:31.556659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:31.557247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:31.557247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:31.558585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 539/1081] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:31.564217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f6cc-79fa-4d65-ace8-ac16001bb3a1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:31.564637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f6cc-79fa-4d65-ace8-ac16001bb3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:31.564778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7f6cc-79fa-4d65-ace8-ac16001bb3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:31.646697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c9f7f6cc-79fa-4d65-ace8-ac16001bb3a1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:31.647867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c9f7f6cc-79fa-4d65-ace8-ac16001bb3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:31.755428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 543/1082] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:32.041090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:32.042607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:32.189934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:32.205376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efeae2-b951-43ea-b1ba-aef7042d45d8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:32.329290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:32.329989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:32.330275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:32.330540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:32.332232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:32.437443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:32.439434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe86d-d82b-4b6e-9e40-0e2387bb18e0 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:32.450508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:32.460013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 540/1083] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:34.669164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:34.669767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:34.669767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:34.671656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 544/1084] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:34.680033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eda72-1dfc-46ec-9174-0f06c963469b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:34.680582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eda72-1dfc-46ec-9174-0f06c963469b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:34.680800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25eda72-1dfc-46ec-9174-0f06c963469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:34.753389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f25eda72-1dfc-46ec-9174-0f06c963469b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:34.754682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f25eda72-1dfc-46ec-9174-0f06c963469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:34.863331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 541/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:35.154301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:35.155868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:35.158751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:35.315307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdcfb27-629f-444e-9c2b-abd4337009f5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:35.393277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:35.438686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:35.439446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:35.439782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:35.440109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:35.441974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:35.544531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:35.546158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b66b4-977a-4898-a5c6-c36e1a826f62 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:35.564769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 545/1086] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:44.861714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=67 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:45.562379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=61 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:49.023245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7d5-5178-4fbf-9e66-1d1c00d140a3 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:49.023793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7d5-5178-4fbf-9e66-1d1c00d140a3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:49.023793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205d7d5-5178-4fbf-9e66-1d1c00d140a3 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:49.025149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 542/1087] 10.222.0.36 () {54 vars in 918 bytes} [Tue Oct 25 13:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:49.033224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dafcea-244c-4ba1-8785-67e08403282b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:49.033741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dafcea-244c-4ba1-8785-67e08403282b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:49.033972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dafcea-244c-4ba1-8785-67e08403282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:49.117785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d4dafcea-244c-4ba1-8785-67e08403282b None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:49.118999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d4dafcea-244c-4ba1-8785-67e08403282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:49.243612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 546/1088] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 25 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:49.258199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62684-346b-4e4c-a4fb-f6138e124476 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:49.258480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62684-346b-4e4c-a4fb-f6138e124476 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:49.258754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a62684-346b-4e4c-a4fb-f6138e124476 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:49.260159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 543/1089] 10.222.0.36 () {54 vars in 918 bytes} [Tue Oct 25 13:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:49.264726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69b1f-eb55-4c05-bbc6-d73d3c152564 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:49.265010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69b1f-eb55-4c05-bbc6-d73d3c152564 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:49.265278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c69b1f-eb55-4c05-bbc6-d73d3c152564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:49.339034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:49.342078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-60c69b1f-eb55-4c05-bbc6-d73d3c152564 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:49.343736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60c69b1f-eb55-4c05-bbc6-d73d3c152564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:49.469774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 547/1090] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 25 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:52.054949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:52.055917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:52.055917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:52.057995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 544/1091] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:17:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:52.066844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a2ec-0836-4968-b01d-e4303fc6eb55 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:52.067289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a2ec-0836-4968-b01d-e4303fc6eb55 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:52.067486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7a2ec-0836-4968-b01d-e4303fc6eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:52.141650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6ee7a2ec-0836-4968-b01d-e4303fc6eb55 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:52.142870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ee7a2ec-0836-4968-b01d-e4303fc6eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:52.200377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:52.244907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 548/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:52.532953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:52.535191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:52.687992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2a7a-b78e-483d-a06b-79792e1055b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:52.767801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:52.813006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:52.813624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:52.813903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:52.814172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:52.815805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:52.913600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:52.915213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c0eea-4f86-498f-8b97-ef14ea7504f5 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:52.933298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 545/1093] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:54.055065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:54.056062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:54.056062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:54.058171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 549/1094] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:54.063260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e475-5475-497e-9070-8a3bdeec7811 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:54.063682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e475-5475-497e-9070-8a3bdeec7811 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:54.063772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e475-5475-497e-9070-8a3bdeec7811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:54.144740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9521e475-5475-497e-9070-8a3bdeec7811 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:54.145975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9521e475-5475-497e-9070-8a3bdeec7811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:54.174327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:54.251203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 546/1095] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:54.416875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:54.418935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:54.574276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f145a69-fb04-47ea-95ab-2ce8f753f10a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:54.616844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:54.692399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:54.692932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:54.693204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:54.693466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:54.695052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:54.787379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:54.788991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6536d19-b8f3-4aa2-9858-5cfdeaba64bc service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:54.807235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 550/1096] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:55.917453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:55.918271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:55.918271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:55.920369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 547/1097] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:17:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:55.925952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e93f7-a2e6-4429-8904-7abbf09697c4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:55.926522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e93f7-a2e6-4429-8904-7abbf09697c4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:55.926630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e93f7-a2e6-4429-8904-7abbf09697c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:55.980915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:56.003305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2c4e93f7-a2e6-4429-8904-7abbf09697c4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:56.004545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2c4e93f7-a2e6-4429-8904-7abbf09697c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:56.105370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 551/1098] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:56.252990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:56.255091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:56.408173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353214f-b2dc-4ba8-b38f-7c2d2a418561 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:56.420650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:56.531969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:56.532511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:56.532784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:56.533053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:56.534659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:56.629843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:56.631498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bf4e2-db24-46c5-87c5-8e8ce22f3fde service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:56.650251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 548/1099] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:57.743458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:57.744064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:57.744064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:57.745358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 552/1100] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:57.751492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc589f3-3578-4027-9761-81caa1e33319 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:57.752217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc589f3-3578-4027-9761-81caa1e33319 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:57.752381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc589f3-3578-4027-9761-81caa1e33319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:57.775009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:57.840261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ffc589f3-3578-4027-9761-81caa1e33319 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:57.841417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ffc589f3-3578-4027-9761-81caa1e33319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:57.944244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 549/1101] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:58.096343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:58.098632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:58.221995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:58.258656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f9813-1d38-4671-b3b4-e668e03d7162 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:17:58.384275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:17:58.384810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:58.385088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:58.385356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:58.386939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:58.466431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:17:58.480546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:17:58.482139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3eeeeb-1c37-4fc2-aec0-2dd0e334f958 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:17:58.500201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 553/1102] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:17:59.602324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:59.602945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:59.602945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:59.604289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 550/1103] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:17:59.610381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9529af-4e84-43cc-9dfc-2b63c63c6e13 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:17:59.611219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9529af-4e84-43cc-9dfc-2b63c63c6e13 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:17:59.611219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9529af-4e84-43cc-9dfc-2b63c63c6e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:17:59.697738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1c9529af-4e84-43cc-9dfc-2b63c63c6e13 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:17:59.698943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1c9529af-4e84-43cc-9dfc-2b63c63c6e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:17:59.804531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 554/1104] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:17:59.957481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:17:59.958926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:00.051178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:00.116030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5439dcb-81d3-4a1a-b234-cfaff657f835 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:00.235265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:00.235821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:00.236101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:00.236375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:00.237979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:00.287131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:00.331936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:00.333577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ba4d5-f564-41d5-a512-aea4599f5f37 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:00.352160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 551/1105] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:01.408076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:01.408589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:01.408589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:01.409994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 555/1106] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:01.415952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6043e5-8444-49eb-aa40-49d18d373155 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:01.416372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6043e5-8444-49eb-aa40-49d18d373155 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:01.416486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6043e5-8444-49eb-aa40-49d18d373155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:01.503096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5e6043e5-8444-49eb-aa40-49d18d373155 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:01.504343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e6043e5-8444-49eb-aa40-49d18d373155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:01.603550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 552/1107] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:18:01.753455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:01.755026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:01.808193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:01.910018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814acab7-89c6-401c-952e-df77bbeb2a0e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:02.034129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:02.034664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:02.034946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:02.035253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:02.036943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:02.043956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:02.130068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:02.131734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe113b90-7a6a-4f20-8a37-8eeb4baf7589 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:02.150058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 556/1108] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:03.193659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:03.194260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:03.194260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:03.195527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 553/1109] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:18:03.200675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abc272-a1ce-44c8-a041-37467d1c71cf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:03.201086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abc272-a1ce-44c8-a041-37467d1c71cf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:03.201194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abc272-a1ce-44c8-a041-37467d1c71cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:03.288517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d4abc272-a1ce-44c8-a041-37467d1c71cf None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:03.289720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d4abc272-a1ce-44c8-a041-37467d1c71cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:03.359947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:03.389503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 557/1110] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:03.537230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:03.538740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:03.557544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:03.696337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b679fd-7ed0-4afb-bb0c-aa43af07033c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:03.790731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:03.817355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:03.817966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:03.818240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:03.818484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:03.820101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:03.919425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:03.921041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b50cbc5-f30d-4244-a276-df8cfe431248 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:03.940575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 554/1111] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:04.962355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:04.962932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:04.962932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:04.964203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 558/1112] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:04.970263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ec5d-3f48-4b6f-b246-9bc3e5b40f40 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:04.970989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ec5d-3f48-4b6f-b246-9bc3e5b40f40 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:04.971196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ec5d-3f48-4b6f-b246-9bc3e5b40f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:05.060020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c544ec5d-3f48-4b6f-b246-9bc3e5b40f40 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:05.061253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c544ec5d-3f48-4b6f-b246-9bc3e5b40f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:05.104165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:05.163039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 555/1113] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:05.317605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:05.320182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:05.489618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f124cc4-40be-4bdb-a178-bbbf7eaff374 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:05.546268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:05.612045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:05.612573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:05.612847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:05.613115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:05.614834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:05.712462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:05.714077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc8ef-0040-47cd-ae18-eb19d60280af service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:05.732861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 559/1114] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:06.761407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:06.762308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:06.762308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:06.764431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 556/1115] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:18:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:06.772205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b0cf-435b-46fe-a610-f4aeb19ceb2a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:06.772613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b0cf-435b-46fe-a610-f4aeb19ceb2a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:06.772893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b0cf-435b-46fe-a610-f4aeb19ceb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:06.852238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0415b0cf-435b-46fe-a610-f4aeb19ceb2a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:06.853437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0415b0cf-435b-46fe-a610-f4aeb19ceb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:06.855588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:06.956115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 560/1116] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:07.105799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:07.108410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:07.262102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fc38b-08d4-4cb0-8346-4382d009e033 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:07.301470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:07.395850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:07.396457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:07.396732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:07.396992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:07.398573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:07.503281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:07.505206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fe419-0047-4c0e-9cdb-2a8ecce45a70 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:07.523425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 557/1117] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:09.545176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:09.545800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:09.545800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:09.547068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 561/1118] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:09.553589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777535d-7916-4529-94f9-ab216aa2b00e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:09.554353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777535d-7916-4529-94f9-ab216aa2b00e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:09.554520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777535d-7916-4529-94f9-ab216aa2b00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:09.606556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:09.635755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8777535d-7916-4529-94f9-ab216aa2b00e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:09.636994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8777535d-7916-4529-94f9-ab216aa2b00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:09.738289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 558/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:10.021331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:10.022877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:10.136084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a114f5d-80f1-40d3-8081-8dcf2c1332f9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:10.168547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:10.212721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:10.213336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:10.213614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:10.213882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:10.215524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:10.327728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:10.329081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356b90-3767-45d7-b58c-0bb5d4b98d96 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:10.348370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 562/1120] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:12.636073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:12.636813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:12.636931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:12.638910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 559/1121] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:12.647477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac9b5-31c2-41a9-883d-d3b96ace9db7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:12.647947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac9b5-31c2-41a9-883d-d3b96ace9db7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:12.648061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ac9b5-31c2-41a9-883d-d3b96ace9db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:12.722147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1f9ac9b5-31c2-41a9-883d-d3b96ace9db7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:12.723455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f9ac9b5-31c2-41a9-883d-d3b96ace9db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:12.736589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:12.823245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 563/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:18:13.110678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:13.113382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:13.233183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18f84-37bd-4c84-a00b-434f44e1a43c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:13.246505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:13.307988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:13.308588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:13.308857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:13.309122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:13.310693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:13.429288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:13.430730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253a822-9a59-47b1-9605-aa7493dc350b service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:13.450188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 560/1123] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:15.908214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071376c-8527-48ff-9753-a68718139759 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:15.908791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071376c-8527-48ff-9753-a68718139759 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:15.908791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071376c-8527-48ff-9753-a68718139759 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:15.910065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 564/1124] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:15.915736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0b66-cd17-48a5-997b-5da917ab5cd2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:15.916214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0b66-cd17-48a5-997b-5da917ab5cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:15.916318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0b66-cd17-48a5-997b-5da917ab5cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:15.967895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:15.986862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-af1e0b66-cd17-48a5-997b-5da917ab5cd2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:15.988127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-af1e0b66-cd17-48a5-997b-5da917ab5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:16.087669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 561/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:16.368407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071376c-8527-48ff-9753-a68718139759 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:16.369969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8071376c-8527-48ff-9753-a68718139759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:16.479457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071376c-8527-48ff-9753-a68718139759 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:16.520410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:16.558398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:16.559003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:16.559310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:16.559583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:16.561182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:16.673108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:16.674555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459d0b4-c8ed-4c53-969c-e5ceb47a940f service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:16.692784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 565/1126] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:18.921749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:18.922332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:18.922440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:18.924004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 562/1127] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:18.929709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5fa9-f13c-4076-814a-d42ee9fcf52d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:18.930133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5fa9-f13c-4076-814a-d42ee9fcf52d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:18.930225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b5fa9-f13c-4076-814a-d42ee9fcf52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:19.007487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fc4b5fa9-f13c-4076-814a-d42ee9fcf52d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:19.008706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fc4b5fa9-f13c-4076-814a-d42ee9fcf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:19.030002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:19.109426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 566/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:19.407105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:19.408698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:19.516855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443bcb2-30f0-4e0e-9bfe-3f2915d8e292 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:19.538500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:19.595538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:19.596166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:19.596448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:19.596723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:19.598330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:19.723082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:19.724545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a46f3-4f0e-493b-99a1-867d3415a3e1 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:19.743782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 563/1129] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:18:21.917653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:21.918479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:21.918479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:21.920500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 567/1130] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:21.928904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218072fc-b611-471f-9fda-283814e8c06b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:21.929326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218072fc-b611-471f-9fda-283814e8c06b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:21.929419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218072fc-b611-471f-9fda-283814e8c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:21.994590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:21.997307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-218072fc-b611-471f-9fda-283814e8c06b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:21.998944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-218072fc-b611-471f-9fda-283814e8c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:22.098854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 564/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:18:22.382597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:22.384271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:22.494575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8dd04-baf6-4d7a-b8fe-d9641e6ae284 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:22.546238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:22.573263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:22.573906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:22.574182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:22.574480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:22.576117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:22.689830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:22.691323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12647918-7961-4e4b-ac16-dfe0323c693d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:22.709275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 568/1132] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:24.889584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:24.890416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:24.890416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:24.892618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 565/1133] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:24.900755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b141f0-6875-4aca-b289-2293492cd37f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:24.901188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b141f0-6875-4aca-b289-2293492cd37f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:24.901307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b141f0-6875-4aca-b289-2293492cd37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:24.971531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-69b141f0-6875-4aca-b289-2293492cd37f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:24.972742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-69b141f0-6875-4aca-b289-2293492cd37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:25.001865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:25.074974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 569/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:18:25.359959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:25.362378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:25.485561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b190a-ee0b-4d7b-bc15-156f096900cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:25.514866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:25.561160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:25.561767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:25.562043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:25.562309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:25.563936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:25.679175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:25.680652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c65c5-d4c3-4c6a-a068-5e59fca2ca3e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:25.699616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 566/1135] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:27.945159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be35-9665-4636-93b7-96963387a75e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:27.945728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be35-9665-4636-93b7-96963387a75e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:27.945728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be35-9665-4636-93b7-96963387a75e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:27.947706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 570/1136] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:27.956632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad358c-88f4-463f-8911-3ea2aed387e4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:27.957059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad358c-88f4-463f-8911-3ea2aed387e4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:27.957240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad358c-88f4-463f-8911-3ea2aed387e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:28.033277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5dad358c-88f4-463f-8911-3ea2aed387e4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:28.034481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5dad358c-88f4-463f-8911-3ea2aed387e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:28.043076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:28.141974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 567/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:28.425820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660be35-9665-4636-93b7-96963387a75e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:28.428002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0660be35-9665-4636-93b7-96963387a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:28.559988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660be35-9665-4636-93b7-96963387a75e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:28.625406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:28.650843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:28.651716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:28.652027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:28.652343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:28.654837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:28.796713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:28.798181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0382aa-1140-4699-b18d-3f9c9bae0e98 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:28.818975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 571/1138] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:31.019391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:31.020289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:31.020289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:31.021602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 568/1139] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:31.027164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039740-6b72-4f7a-a9a9-c1e8399b119d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:31.027601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039740-6b72-4f7a-a9a9-c1e8399b119d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:31.027705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039740-6b72-4f7a-a9a9-c1e8399b119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:31.096141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4c039740-6b72-4f7a-a9a9-c1e8399b119d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:31.097360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4c039740-6b72-4f7a-a9a9-c1e8399b119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:31.136274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:31.197025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 572/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:31.475287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:31.477825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:31.592742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951b874-797b-4267-a9b2-a05d62e01988 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:31.633821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:31.673105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:31.673651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:31.673932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:31.674202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:31.675845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:31.805324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:31.806774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08460e-7730-4ab7-9f55-1cad420e27ae service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:31.826008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 569/1141] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:33.973121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:33.973681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:33.973681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:33.974941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 573/1142] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:33.982700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b3bc-4715-45cc-ad49-17ce132cf849 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:33.983175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b3bc-4715-45cc-ad49-17ce132cf849 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:33.983287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11b3bc-4715-45cc-ad49-17ce132cf849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:34.055055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8b11b3bc-4715-45cc-ad49-17ce132cf849 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:34.056305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b11b3bc-4715-45cc-ad49-17ce132cf849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:34.074699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:34.157896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 570/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:34.437588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:34.440137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:34.551530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637163-ae4c-4080-9bc2-bb32a55e7d01 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:34.623612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:34.631376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:34.631997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:34.632273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:34.632540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:34.634137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:34.748654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:34.750105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125b8aa-7f31-4897-b5be-2638aec85aaa service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:34.768813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 574/1144] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:36.908045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:36.908896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:36.908896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:36.911086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 571/1145] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:18:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:36.916641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51f55-2001-4747-96f4-100964473774 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:36.917104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51f55-2001-4747-96f4-100964473774 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:36.917216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51f55-2001-4747-96f4-100964473774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:36.985351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b7b51f55-2001-4747-96f4-100964473774 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:36.986577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b7b51f55-2001-4747-96f4-100964473774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:37.037708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:37.091768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 575/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:37.374727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:37.377243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:37.489100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfcd51-655a-492c-b452-2e2cc4a5f4e7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:37.539422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:37.566622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:37.567330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:37.567656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:37.567942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:37.569579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:37.681361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:37.682822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a26ca1-edf0-4c00-b6df-27a1a3982454 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:37.701269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 572/1147] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:18:47.090047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=22 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:47.698806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=66 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:57.449464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc66dc-5e9c-4c1a-a3fd-cd4da1e4c187 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:57.450031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc66dc-5e9c-4c1a-a3fd-cd4da1e4c187 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:57.450031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc66dc-5e9c-4c1a-a3fd-cd4da1e4c187 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:57.451359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 576/1148] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 25 13:18:57.457760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6e1c-5b85-41f1-a4e2-3d7eac5b3530 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:57.458200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6e1c-5b85-41f1-a4e2-3d7eac5b3530 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:57.458290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6e1c-5b85-41f1-a4e2-3d7eac5b3530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:57.534134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-890e6e1c-5b85-41f1-a4e2-3d7eac5b3530 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:57.535420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-890e6e1c-5b85-41f1-a4e2-3d7eac5b3530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:57.639800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 573/1149] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:57.936007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:57.936289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:57.936562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:57.937935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 577/1150] 10.222.0.36 () {54 vars in 962 bytes} [Tue Oct 25 13:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:18:57.943270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895410e-6898-47d9-b181-f1e5c2361a15 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:57.943562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895410e-6898-47d9-b181-f1e5c2361a15 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:57.943836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895410e-6898-47d9-b181-f1e5c2361a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:58.025337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c895410e-6898-47d9-b181-f1e5c2361a15 None None] MFA Rules not processed for user `7e49839dc5bc47d6a9a91800fef249c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:18:58.026552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c895410e-6898-47d9-b181-f1e5c2361a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:58.048404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:58.106233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 574/1151] 10.222.0.36 () {58 vars in 1059 bytes} [Tue Oct 25 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:18:58.117165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:18:58.118707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:58.228411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dcefd-03f6-4fb4-ba96-c0597fb0b9fb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:18:58.305027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:18:58.305622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:18:58.305894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:18:58.306160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:18:58.307767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:18:58.363912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:18:58.426253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:18:58.427739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdddfe-b543-47d7-8b8c-3ead762fbbf6 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:18:58.447673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 578/1152] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:00.673905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:00.674431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:00.674431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:00.675893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 575/1153] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:00.682480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e54247-e228-44e9-adba-af6ed6d14914 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:00.683232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e54247-e228-44e9-adba-af6ed6d14914 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:00.683468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e54247-e228-44e9-adba-af6ed6d14914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:00.754901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d0e54247-e228-44e9-adba-af6ed6d14914 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:00.756173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d0e54247-e228-44e9-adba-af6ed6d14914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:00.857710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 579/1154] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:19:01.129876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:01.132294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:01.264755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d00bc-b46b-4fc0-9920-3fe14917331c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:01.317346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:01.363214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:01.363824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:01.364099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:01.364365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:01.365007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:01.366167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:19:01.366167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:19:01.366167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:19:01.367340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd49dd3-b989-4815-9b64-18c8fa039dca admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:01.388264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 576/1155] 10.222.0.36 () {56 vars in 1108 bytes} [Tue Oct 25 13:19:01 2022] GET /identity/v3/projects => generated 2553 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:19:03.521933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:03.522914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:03.522914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:03.524916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 580/1156] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:03.532614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3978-e5d5-48bf-a5eb-f04c38b2999c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:03.533057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3978-e5d5-48bf-a5eb-f04c38b2999c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:03.533177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3978-e5d5-48bf-a5eb-f04c38b2999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:03.601553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3b7c3978-e5d5-48bf-a5eb-f04c38b2999c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:03.602746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b7c3978-e5d5-48bf-a5eb-f04c38b2999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:03.704395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 577/1157] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:19:03.999236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:04.000911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:04.012043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:04.155457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5b38-6a7d-44e4-bdd8-6791229090ee None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:04.243681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:04.276325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:04.276929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:04.277197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:04.277467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:04.279083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:04.377715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:04.379383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05934-b36c-438f-9b35-3b5461bc1d73 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:04.398889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 581/1158] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:06.725988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:06.726902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:06.726902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:06.729110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 578/1159] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:06.734662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f94e8-5faf-4665-877b-e212aaecc987 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:06.735097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f94e8-5faf-4665-877b-e212aaecc987 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:06.735229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f94e8-5faf-4665-877b-e212aaecc987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:06.802369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b84f94e8-5faf-4665-877b-e212aaecc987 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:06.803612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b84f94e8-5faf-4665-877b-e212aaecc987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:06.852914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:06.914316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 582/1160] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:19:07.210753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:07.212340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:07.237276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:07.321035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920b537-1b3a-4209-8fb1-70eb7a7b1dec None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:07.397966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:07.398616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:07.398894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:07.399174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:07.400813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:07.489804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:07.526619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:07.528126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ce253-f693-41ea-904c-1171c7ba8427 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:07.548086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 579/1161] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:09.748398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:09.748978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:09.748978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:09.750336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 583/1162] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:09.756812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f6ff-951d-4189-b324-7311d8a77805 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:09.757277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f6ff-951d-4189-b324-7311d8a77805 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:09.757376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9f6ff-951d-4189-b324-7311d8a77805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:09.837721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ddc9f6ff-951d-4189-b324-7311d8a77805 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:09.838966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ddc9f6ff-951d-4189-b324-7311d8a77805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:09.946378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 580/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:19:10.232980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:10.235534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:10.346721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb43b01-a9fb-4347-a539-832775ce4299 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:10.422645 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:10.424150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:10.425169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:10.425430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:10.425754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:10.428341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:10.542115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:10.543600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d2993-6847-4dc2-bca8-5d2602ba0062 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:10.562128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 584/1164] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:12.860265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:12.861075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:12.861075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:12.863313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 581/1165] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:12.869862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343a9a3-f184-4e67-bd6a-b952075e9434 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:12.870314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343a9a3-f184-4e67-bd6a-b952075e9434 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:12.870447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343a9a3-f184-4e67-bd6a-b952075e9434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:12.944661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2343a9a3-f184-4e67-bd6a-b952075e9434 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:12.945776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2343a9a3-f184-4e67-bd6a-b952075e9434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:13.001059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:13.045810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 585/1166] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:19:13.336222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:13.338781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:13.382116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:13.450884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8e61b-01a3-4f31-8361-5c2193300d80 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:13.529068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:13.529643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:13.529934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:13.530203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:13.531826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:13.613781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:13.648081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:13.649546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa2e01-a783-4575-8cd9-5ce1fe8b3cc3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:13.668122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 582/1167] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:16.033050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:16.034005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:16.034005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:16.036127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 586/1168] 10.222.0.36 () {54 vars in 927 bytes} [Tue Oct 25 13:19:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:19:16.042618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b1029-bccf-4a56-a9cf-f4ee749fb17f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:16.043349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b1029-bccf-4a56-a9cf-f4ee749fb17f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:16.043511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b1029-bccf-4a56-a9cf-f4ee749fb17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:16.119354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b90b1029-bccf-4a56-a9cf-f4ee749fb17f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:19:16.120567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b90b1029-bccf-4a56-a9cf-f4ee749fb17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:16.225328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 583/1169] 10.222.0.36 () {58 vars in 1024 bytes} [Tue Oct 25 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:19:16.516545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:19:16.518996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:16.638151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06965cd1-f6cf-4c8e-94dc-b4142b1e9844 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:19:16.716472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:19:16.717081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:19:16.717355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:19:16.717653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:19:16.719213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:19:16.727761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:16.836771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:19:16.838230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a20549-54a3-4cb3-9452-c1db7a1f9bf1 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:19:16.857472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 587/1170] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:19:26.223904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=82 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:19:26.854804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=50 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:35.804190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:35.805053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:35.805053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:35.895080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:20:35.896351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.003481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 584/1171] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:20:36.012045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:36.012586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:36.012586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:36.103452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:20:36.104680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.204185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 588/1172] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:20:36.210103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:36.211866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.223856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8cf92-5155-410c-a1da-876f5a31eb43 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:36.294711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:36.330420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:36.331023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:36.331358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:36.331622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:36.332641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:20:36.332943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:36.333965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:20:36.333965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:20:36.333965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:20:36.334856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:36.346239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 585/1173] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:20:36.356238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:36.365712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad382d9-857f-4588-9541-3fd17014f1b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:36.436021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:36.470545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:36.471080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:36.471410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:36.471674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:36.472300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:36.473322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:20:36.473322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:20:36.473322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:20:36.474509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:36.534202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 589/1174] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:20:36 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:20:36.538742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:36.540482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.574394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c21bc-705a-40f4-8c73-ad69107cefb8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:36.670775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:36.671481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:36.671754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:36.672015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:36.672643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:36.673650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:20:36.673650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:20:36.673650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:20:36.674836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:36.718697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 586/1175] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:20:36 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:20:36.723169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:36.724767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.791772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21edfd-d8b8-4841-9869-f8778236b14b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:36.878731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:36.889849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:36.890460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:36.890735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:36.890999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:36.891698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:36.893305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:36.905714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 590/1176] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:20:36.910042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:36.911706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:36.923586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258c2a4-d931-49b7-850f-6f3c460fcf75 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:36.950193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:37.025016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:37.025639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.025918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.026189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.026845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:37.028512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36eca-0ead-48fc-b654-7686fb1968b4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:37.041506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 587/1177] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:20:37.046112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:37.047682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:37.083670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93883-c4f2-4d16-8c67-9f210bd5fa1b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:37.196111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:37.196758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.197027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.197298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] PATH_INFO: `/v3/projects/544543e306c94be0823a108aaea7fe30/users/689c58dee81d47bc98d401f62dd2211f/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.221411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544543e306c94be0823a108aaea7fe30, user_id=689c58dee81d47bc98d401f62dd2211f, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:37.224188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6429-83bc-4801-bc5a-61dcbc1e8a79 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:37.251964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 591/1178] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:20:37 2022] PUT /identity/v3/projects/544543e306c94be0823a108aaea7fe30/users/689c58dee81d47bc98d401f62dd2211f/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:20:37.263471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ba3e-09e4-448f-affd-44f70e7c445c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.263762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ba3e-09e4-448f-affd-44f70e7c445c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.264031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ba3e-09e4-448f-affd-44f70e7c445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.322543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5232ba3e-09e4-448f-affd-44f70e7c445c None None] MFA Rules not processed for user `689c58dee81d47bc98d401f62dd2211f`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:20:37.323728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5232ba3e-09e4-448f-affd-44f70e7c445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:37.404898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 588/1179] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:20:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:20:37.418568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.418855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.419128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.420520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 592/1180] 10.222.0.36 () {54 vars in 963 bytes} [Tue Oct 25 13:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:20:37.425286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c755-ec45-4e90-8200-48224b7dfea8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.425573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c755-ec45-4e90-8200-48224b7dfea8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.425848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c755-ec45-4e90-8200-48224b7dfea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.459469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:37.510858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6917c755-ec45-4e90-8200-48224b7dfea8 None None] MFA Rules not processed for user `7623623b296e4d04949973891b697c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:20:37.512099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6917c755-ec45-4e90-8200-48224b7dfea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:37.589985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 589/1181] 10.222.0.36 () {58 vars in 1060 bytes} [Tue Oct 25 13:20:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:20:37.602362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:37.604646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:37.660107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:37.715773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79948-9496-4dfb-9f61-ed90597a753a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:37.800719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:37.801581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.801966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.802275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.804870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:37.908039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:37.909407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:37.912908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:37.928792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 593/1182] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:20:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:20:37.979647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cfb9e-b0e5-4de7-bf41-d8a40227cbf7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:37.980141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cfb9e-b0e5-4de7-bf41-d8a40227cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:37.980141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cfb9e-b0e5-4de7-bf41-d8a40227cbf7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:37.981458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 590/1183] 10.222.0.36 () {52 vars in 830 bytes} [Tue Oct 25 13:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:20:37.985241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:37.986616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:38.099731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a385-881a-4122-9eb7-3af6c38f4e40 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:38.194782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:38.195365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:38.195635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:38.195900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] PATH_INFO: `/v3/users/689c58dee81d47bc98d401f62dd2211f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:38.209461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=689c58dee81d47bc98d401f62dd2211f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:38.210491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:20:38.210491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:20:38.210491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:20:38.211739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:38.221498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:38.293169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '689c58dee81d47bc98d401f62dd2211f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:38.317123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '689c58dee81d47bc98d401f62dd2211f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:38.317567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '689c58dee81d47bc98d401f62dd2211f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:38.332478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '689c58dee81d47bc98d401f62dd2211f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:38.333034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '689c58dee81d47bc98d401f62dd2211f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:38.350961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 594/1184] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:20:37 2022] DELETE /identity/v3/users/689c58dee81d47bc98d401f62dd2211f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:20:38.355670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:38.355968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:38.356247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:38.417353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:20:38.418572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:38.495869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:38.508835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 591/1185] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:20:38.520187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:38.521428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:38.628608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67fd79-39bc-4e30-bfe1-77bdd0feb0cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:38.686553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:38.712437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:38.713063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:38.713346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:38.713620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:38.715287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:38.829928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:38.831398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a43425-6608-40a6-9e8d-9c93530533d8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:38.850764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 595/1186] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:20:39.183838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:20:39.185073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:20:39.197108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f4cd8-de59-4fe9-8755-1bf25033458e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:20:39.297718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:20:39.298343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:20:39.298616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:20:39.298881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] PATH_INFO: `/v3/projects/544543e306c94be0823a108aaea7fe30` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:20:39.309189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=544543e306c94be0823a108aaea7fe30)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:20:39.310226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:20:39.310226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:20:39.310226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:20:39.311460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:20:39.371396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544543e306c94be0823a108aaea7fe30', 'request_id': 'req-db6b59da-99a3-4b40-ae73-f1f041e09bdb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:39.384756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544543e306c94be0823a108aaea7fe30', 'request_id': 'req-db6b59da-99a3-4b40-ae73-f1f041e09bdb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:39.395651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544543e306c94be0823a108aaea7fe30', 'request_id': 'req-db6b59da-99a3-4b40-ae73-f1f041e09bdb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:39.419819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] The token cache is being invalidate because project 544543e306c94be0823a108aaea7fe30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:20:39.420134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:39.420491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db6b59da-99a3-4b40-ae73-f1f041e09bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:20:39.422018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 592/1187] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:20:39 2022] DELETE /identity/v3/projects/544543e306c94be0823a108aaea7fe30 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:20:48.848245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=62 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:20:49.417615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:22:08.287197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:08.287865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:08.287865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:08.289238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 596/1188] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:08.294400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba3d6-fd87-4641-b61c-0c0874cf983c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:08.294854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba3d6-fd87-4641-b61c-0c0874cf983c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:08.294959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba3d6-fd87-4641-b61c-0c0874cf983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:08.378374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-754ba3d6-fd87-4641-b61c-0c0874cf983c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:08.379636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-754ba3d6-fd87-4641-b61c-0c0874cf983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:08.488846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 593/1189] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:08.642545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:08.645781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:08.779591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e525f3-245b-4356-9b46-5d4e5cd81936 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:08.786566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:08.787280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:08.787579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:08.787852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:08.796574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:08.798191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892a854-c8f2-4382-a8d9-acf9bc8fdd74 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:08.808505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 597/1190] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:09.649154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46483d-e069-4479-abc3-cd255051922f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:09.649651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46483d-e069-4479-abc3-cd255051922f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:09.649651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46483d-e069-4479-abc3-cd255051922f None None] PATH_INFO: `/v3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:09.651007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 594/1191] 10.222.0.26 () {54 vars in 925 bytes} [Tue Oct 25 13:22:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:22:09.658057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53378fa0-40e3-4ef7-bb15-d9bb1bfb4db9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:09.658516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53378fa0-40e3-4ef7-bb15-d9bb1bfb4db9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:09.658628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53378fa0-40e3-4ef7-bb15-d9bb1bfb4db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:09.726008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-53378fa0-40e3-4ef7-bb15-d9bb1bfb4db9 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:09.727998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-53378fa0-40e3-4ef7-bb15-d9bb1bfb4db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:09.802307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:22:09.811101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 598/1192] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 25 13:22:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:09.877666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9355-8925-4db3-a5ee-72a8820c7c20 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:09.878044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9355-8925-4db3-a5ee-72a8820c7c20 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:09.878225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9355-8925-4db3-a5ee-72a8820c7c20 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:09.879718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 595/1193] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:09.884748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334189-2893-4540-8163-aefabf439d29 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:09.885224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334189-2893-4540-8163-aefabf439d29 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:09.885665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334189-2893-4540-8163-aefabf439d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:09.906076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5e334189-2893-4540-8163-aefabf439d29 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:09.907362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5e334189-2893-4540-8163-aefabf439d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:09.916471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 599/1194] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:10.071779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf9355-8925-4db3-a5ee-72a8820c7c20 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:10.093034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf9355-8925-4db3-a5ee-72a8820c7c20 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:10.099295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:10.099901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:10.100181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:10.100458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:10.108992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:10.110616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb832d-9340-4671-abf3-cc6cb19316eb service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:10.120658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 596/1195] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:18.834083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9da16-8d63-43a3-a38f-67f2f87995de None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:18.834560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9da16-8d63-43a3-a38f-67f2f87995de None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:18.834664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9da16-8d63-43a3-a38f-67f2f87995de None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:18.836003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 600/1196] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:18.841260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5345e-223e-41a9-9224-41940cb3d914 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:18.841712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5345e-223e-41a9-9224-41940cb3d914 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:18.841825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5345e-223e-41a9-9224-41940cb3d914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:18.863107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e1b5345e-223e-41a9-9224-41940cb3d914 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:18.864349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e1b5345e-223e-41a9-9224-41940cb3d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:18.874271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 597/1197] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:22:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:18.886385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9da16-8d63-43a3-a38f-67f2f87995de None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:18.906141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9da16-8d63-43a3-a38f-67f2f87995de None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:18.912168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:18.912769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:18.913034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:18.913316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:18.921741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:18.923352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af9766-6d78-4864-983b-91ed0463912b service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:18.933088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 601/1198] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:21.584827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6604-2f12-44f2-bb7b-9c43fdae8c6f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:21.585446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6604-2f12-44f2-bb7b-9c43fdae8c6f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:21.585446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b6604-2f12-44f2-bb7b-9c43fdae8c6f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:21.586858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 598/1199] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:21.591878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08feae1-2a8d-4393-8846-dd9eb7165fd1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:21.592308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08feae1-2a8d-4393-8846-dd9eb7165fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:21.592410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08feae1-2a8d-4393-8846-dd9eb7165fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:21.613198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e08feae1-2a8d-4393-8846-dd9eb7165fd1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:21.614411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e08feae1-2a8d-4393-8846-dd9eb7165fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:21.623520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 602/1200] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:21.778664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6604-2f12-44f2-bb7b-9c43fdae8c6f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:21.794488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b6604-2f12-44f2-bb7b-9c43fdae8c6f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:21.800238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:21.800895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:21.801177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:21.801447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:21.810213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:21.811845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2adc-1495-404d-b581-fb937d4aec4c service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:21.821417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 599/1201] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:22.879941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ff3a-d7c6-4155-8865-6b9c4e434883 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:22.880791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ff3a-d7c6-4155-8865-6b9c4e434883 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:22.880791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079ff3a-d7c6-4155-8865-6b9c4e434883 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:22.882830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 603/1202] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 25 13:22:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:22.889372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894494e-2de6-4005-878b-6ab00bad5761 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:22.889839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894494e-2de6-4005-878b-6ab00bad5761 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:22.889945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894494e-2de6-4005-878b-6ab00bad5761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:22.910440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3894494e-2de6-4005-878b-6ab00bad5761 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:22.911739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3894494e-2de6-4005-878b-6ab00bad5761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:22.920262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 600/1203] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 25 13:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:22:23.073488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079ff3a-d7c6-4155-8865-6b9c4e434883 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:23.089764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079ff3a-d7c6-4155-8865-6b9c4e434883 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:23.095619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:23.096211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:23.096486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:23.096754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:23.105260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:23.106902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194f5c1-a549-4ca2-80b5-ff622fdca645 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:23.116825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 604/1204] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:26.987577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:26.988113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:26.988113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:26.990219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 601/1205] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:22:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:26.998647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4117-27bc-4131-bce2-3483611cfabf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:26.999056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4117-27bc-4131-bce2-3483611cfabf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:26.999249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4117-27bc-4131-bce2-3483611cfabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:27.020046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b0ff4117-27bc-4131-bce2-3483611cfabf None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:27.021283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b0ff4117-27bc-4131-bce2-3483611cfabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:27.030642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 605/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:27.316290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:27.318373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:27.353692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:22:27.402695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478f7d-f798-4aa2-97e1-bc30537fd534 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:27.406914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:27.407626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:27.407826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:27.408099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:27.418151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:27.419631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb71320-b347-422f-9340-164e48154ae1 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:27.429903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 602/1207] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:22:29.533535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4422d-9066-4b6f-a156-fb21b3547962 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:29.534474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4422d-9066-4b6f-a156-fb21b3547962 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:29.534474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4422d-9066-4b6f-a156-fb21b3547962 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:29.536647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 606/1208] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:22:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:29.542053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc40f-4da9-45c3-9fe6-53853f51435b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:29.542601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc40f-4da9-45c3-9fe6-53853f51435b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:29.542601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc40f-4da9-45c3-9fe6-53853f51435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:29.562635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d77dc40f-4da9-45c3-9fe6-53853f51435b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:29.563844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d77dc40f-4da9-45c3-9fe6-53853f51435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:29.572030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 603/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:29.853040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4422d-9066-4b6f-a156-fb21b3547962 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:29.871340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4422d-9066-4b6f-a156-fb21b3547962 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:29.877064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:29.877656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:29.877931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:29.878196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:29.887495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:29.888928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62649f0f-af75-45f9-b411-fd2d41b669c8 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:29.898273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 607/1210] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:33.549798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:33.550705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:33.550705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:33.552973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 604/1211] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:22:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:33.559344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46a01-5279-4a23-b16c-38a4e23d9931 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:33.560124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46a01-5279-4a23-b16c-38a4e23d9931 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:33.560301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46a01-5279-4a23-b16c-38a4e23d9931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:33.582075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bdf46a01-5279-4a23-b16c-38a4e23d9931 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:33.583346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bdf46a01-5279-4a23-b16c-38a4e23d9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:33.591858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 608/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:33.901890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:33.904060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:33.985393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077eef-63a2-4b7d-a5a5-10945e0c0d9f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:33.989451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:33.990059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:33.990327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:33.990575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:34.000192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:34.001615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291042-dc36-48cc-a97a-3f746650a475 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:34.010981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 605/1213] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:36.137333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:36.138183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:36.138183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:36.140202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 609/1214] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 25 13:22:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:22:36.146382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606db1-4e9e-42e4-9470-631f2903c1cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:36.146814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606db1-4e9e-42e4-9470-631f2903c1cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:36.146921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606db1-4e9e-42e4-9470-631f2903c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:36.170727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4d606db1-4e9e-42e4-9470-631f2903c1cc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:22:36.172040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d606db1-4e9e-42e4-9470-631f2903c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:36.180513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 606/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 25 13:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:22:36.486378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:22:36.489352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:22:36.511687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ab6f-7df5-44b6-90fd-6505596a1dbc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:22:36.517382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:22:36.517988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:22:36.518266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:22:36.518536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:22:36.528256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:22:36.529687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f11aa-14f4-4c19-91ea-29e51a1f3794 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:22:36.539776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 610/1216] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:22:46.165745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=63 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:22:46.535455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=37 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:23:59.881079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db90c9-574b-4def-9647-85d40ff847b3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:23:59.882333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db90c9-574b-4def-9647-85d40ff847b3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:23:59.882847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db90c9-574b-4def-9647-85d40ff847b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:23:59.906733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-26db90c9-574b-4def-9647-85d40ff847b3 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:23:59.908038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-26db90c9-574b-4def-9647-85d40ff847b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:23:59.921174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 607/1217] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:23:59.931903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58fa64-b89c-479d-89c1-c5ffc756ba0b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:23:59.933222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58fa64-b89c-479d-89c1-c5ffc756ba0b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:23:59.933731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58fa64-b89c-479d-89c1-c5ffc756ba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:23:59.940954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:23:59.941330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:23:59.941531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:23:59.962371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0b58fa64-b89c-479d-89c1-c5ffc756ba0b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:23:59.964032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0b58fa64-b89c-479d-89c1-c5ffc756ba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:23:59.972245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:23:59.973429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 611/1218] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:23:59.973570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:23:59.975128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9774a2-e92d-4892-bc0e-31a58cb4d0b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:23:59.979508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9774a2-e92d-4892-bc0e-31a58cb4d0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:23:59.980070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9774a2-e92d-4892-bc0e-31a58cb4d0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:23:59.982178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 608/1219] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:23:59.982942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.007667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d6512-cc95-42a1-9093-f520e3465236 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.021909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cc9774a2-e92d-4892-bc0e-31a58cb4d0b5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.022299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.022964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.023294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.023580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.024464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.024778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.025084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc9774a2-e92d-4892-bc0e-31a58cb4d0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.026610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4c3ed-d882-4539-9335-55d885f4e92c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.043996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 612/1220] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.045753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.046260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.046743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.050704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 609/1221] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.062404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024358d-da13-468a-b4fc-135809248f41 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.062748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024358d-da13-468a-b4fc-135809248f41 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.062963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024358d-da13-468a-b4fc-135809248f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.077139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.078743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.094484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 613/1222] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.095890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.104213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7024358d-da13-468a-b4fc-135809248f41 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.105477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7024358d-da13-468a-b4fc-135809248f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.111528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf296f3-ca9b-4b95-b3d6-96d3323ad347 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.118910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.119862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.120366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.120936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.121541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 610/1223] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:00.122033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.122393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024358d-da13-468a-b4fc-135809248f41 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.127061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.141186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024358d-da13-468a-b4fc-135809248f41 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.152002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.152951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.153207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.153465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.154132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.154433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.156111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.185241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 611/1224] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.191209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.209022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a9d31-6e82-49c8-97b5-bc34c828acc3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.216390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 614/1225] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.217255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.218262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.218901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.219178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.219453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.220113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.220414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.222034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3aa11-5c08-483e-87dc-b6563975f6a0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.246343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 612/1226] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.248730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956532be-6af6-4f5f-ad4a-8e51a05e06cf None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.249014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956532be-6af6-4f5f-ad4a-8e51a05e06cf None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.249283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956532be-6af6-4f5f-ad4a-8e51a05e06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.282978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-956532be-6af6-4f5f-ad4a-8e51a05e06cf None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.284676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-956532be-6af6-4f5f-ad4a-8e51a05e06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.322198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3b00-5bda-47b7-aa1c-6798817cf0b7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.327603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.327947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.328228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.328502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.329264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.331285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.362800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 615/1227] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.363693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.366410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 613/1228] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.367671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.367924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.368200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.388105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.389344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.430423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e3b00-70a4-42ad-933c-02c253cc6569 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.436107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.436722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.436999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.437265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.437903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.438954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:24:00.438954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:24:00.438954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:24:00.440194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb160cf-7854-4c21-a248-fd13d55aa165 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.450536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 614/1229] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.451403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.461761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d47209-9e94-4895-bcdf-14b212c28ca1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.467421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.470618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 616/1230] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.470745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.470745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.470745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.470990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e9c6-dc4a-40c6-832f-dd43101e4581 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.471072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.471176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e9c6-dc4a-40c6-832f-dd43101e4581 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.471176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99e9c6-dc4a-40c6-832f-dd43101e4581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.471350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:24:00.471350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:24:00.471350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:24:00.471551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f498300-7e71-4cbd-b414-c9ffac7f5bd2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.488858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9d99e9c6-dc4a-40c6-832f-dd43101e4581 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.490169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9d99e9c6-dc4a-40c6-832f-dd43101e4581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.499542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 617/1231] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.500808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.501091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.501363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.503683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 615/1232] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.505026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68139e-6593-406c-a807-fcd1ce1434f8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.505292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68139e-6593-406c-a807-fcd1ce1434f8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.505565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68139e-6593-406c-a807-fcd1ce1434f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.522958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.524185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.525922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ec68139e-6593-406c-a807-fcd1ce1434f8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.527528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec68139e-6593-406c-a807-fcd1ce1434f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.537498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:00.541018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:00.588080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 618/1233] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.588911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.591229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 616/1234] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.591229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edf86-8a66-4f0d-a235-1d186268f96a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.591229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edf86-8a66-4f0d-a235-1d186268f96a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.591520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edf86-8a66-4f0d-a235-1d186268f96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.606076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0d42-0017-4118-82a2-042e37cf7c02 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.611020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.611703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.612011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.612257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.612580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-006edf86-8a66-4f0d-a235-1d186268f96a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.612958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.613799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-006edf86-8a66-4f0d-a235-1d186268f96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.614878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.622356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 617/1235] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.623674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.623953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.624228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.641339 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 619/1236] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/users => generated 291 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.642580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.648552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.649934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.652416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea01a32-8be9-4f8d-9fec-8f5f77c1f696 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.656944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.657535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.657811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.658079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.661982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 618/1237] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.662133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.662237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.662336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1afa9-7e12-4862-9622-e3a50dbd87c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.673867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 620/1238] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:00.674040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41d9c9-3dba-4a08-97f6-e68b49e6b31d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.674161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc16f3-0e66-4fef-9f67-cfd11741a7f2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.674161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc16f3-0e66-4fef-9f67-cfd11741a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.674161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc16f3-0e66-4fef-9f67-cfd11741a7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.676541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.677121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.677394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.677661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.678294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.680261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6af02-1f4c-4293-80c3-955f21951273 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.691923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0cdc16f3-0e66-4fef-9f67-cfd11741a7f2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.693114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0cdc16f3-0e66-4fef-9f67-cfd11741a7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.701320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 621/1239] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.702491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.702773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.705953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.706345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 619/1240] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.708291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.708575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.708843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.723812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.724999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.728566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:00.729793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:00.733873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 622/1241] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.735104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.738163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 620/1242] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:00.739048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.748889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585458b9-f713-44f4-b0e8-6b3622f105f4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.749064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e89e-5d75-45f8-bb60-88968e99c223 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.753464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.754052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.754177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.754327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.754594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.754688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.754920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.755221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.755332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.755870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.756168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.756865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.757182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:24:00.757182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:24:00.757182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:24:00.758047 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.770275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 621/1243] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:00.770275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.770568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 623/1244] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.771084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.778238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d5f0d-5a43-4978-ba2d-963aea758f4f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.782874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.783485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.783770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.784039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.784681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.784976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.786545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3589c7-ad14-457c-b5bf-06767494ac90 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.786670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.791897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.792446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.792720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.792985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.793626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.795249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.799242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 622/1245] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.800022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.809745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7005ee0-52e3-4bac-83dc-aede4602dddc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.813322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 624/1246] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:00.814156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.814374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.814962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.815273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.815547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.816319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.816554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.818163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.824051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb1938-9448-4088-95da-0b9075e1f6f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.828838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.829451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.829735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.830086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 623/1247] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.830217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.830735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.830834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.833194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.843998 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d1d2-c77f-499b-bcbd-e5ec804af49e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.844629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 625/1248] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:00.845491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.849224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.849833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.850111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.850503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.851026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.851359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.852955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.855400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3c5fe-ed00-432c-b736-17d95cf4c899 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.860092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.860675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.860955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.861219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.861855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:00.862150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.863746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.873225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 624/1249] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.873225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.876304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 626/1250] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:00.876930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.884716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4918b-6813-486a-ad46-f8995c03e294 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.886248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2c62-2f2c-4e35-bea4-e4e36c57d891 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.890037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.890653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.890928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.891023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.891223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.891532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.891807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.891899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.892071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.892689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.893494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.894541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.904186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 625/1251] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:00.904972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.914818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d559f32-8304-4ef2-b62a-e0b019d97654 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:00.920332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:00.920916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:00.921196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:00.921491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:00.922166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:00.924142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:00.928436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 627/1252] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.929331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.955683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 626/1253] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:00.957528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:00.996019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af1a0-8007-4f51-a3a5-3fd82c06a782 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.006099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.020842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99318c-b038-44a2-9b22-d2430949ef18 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.025664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.026268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.026530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.026850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.027475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.029367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.033844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.057475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 627/1254] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:00 2022] POST /identity/v3/projects => generated 359 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.058347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.058553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.059083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.059391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.059649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] PATH_INFO: `/v3/projects/db46e2a9752d428bbfff5da2ddb16e3a/users/213a6f956e5c48b4b45a6481e29db89c/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.074609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=db46e2a9752d428bbfff5da2ddb16e3a, user_id=213a6f956e5c48b4b45a6481e29db89c, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.077310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.091063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 628/1255] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:00 2022] PUT /identity/v3/projects/db46e2a9752d428bbfff5da2ddb16e3a/users/213a6f956e5c48b4b45a6481e29db89c/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:01.092108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.122250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87948-fd19-4fb2-a696-d645410230a1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.156400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef271e-7965-4b97-a4e6-cdea441690cf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.160933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.161568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.161842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.162093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.162708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.164684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.182325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.183029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.183334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.183632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.184504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.186190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.190830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 629/1256] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.191818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.199953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 628/1257] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.200792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.211518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2af01-1c4e-4a4c-982e-f99a7775d899 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.255655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7153a64-fe51-47d1-a9f5-391ec19afd42 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.260376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.261009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.261315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.261581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.262209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.264191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.269086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.269749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.270023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.270293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] PATH_INFO: `/v3/projects/832915e9a8694e129b533ee5f00cffc2/users/68e23f85e2a244b8bddf47beb3da5da8/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.285663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832915e9a8694e129b533ee5f00cffc2, user_id=68e23f85e2a244b8bddf47beb3da5da8, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.288607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.292249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 630/1258] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.293150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.302768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.306644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 629/1259] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:01 2022] PUT /identity/v3/projects/832915e9a8694e129b533ee5f00cffc2/users/68e23f85e2a244b8bddf47beb3da5da8/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:01.307582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.327054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.358448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439311b-1f70-4bb0-ba3d-d494eed14f1c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.380135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ade91-a1c8-4ea3-8ea4-1506947c0a56 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.386446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.387056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.387795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.388065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.388711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.391445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.419133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 630/1260] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.419133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.419460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.419460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.419460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.419460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.420213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.422556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe87645-f659-4b09-ae2e-4907e6ffac29 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.428298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3ab3a-dcab-4fb2-bdfb-0d962a74f52a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.433545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.434180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.434484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.434781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.435508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.437605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.445289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 631/1261] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.447222 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.447534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.448260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.462863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 631/1262] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/users => generated 280 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.462863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.468534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:01.469738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:01.472213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa99ce-0565-4472-a0ce-95857f57aa39 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.476814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.477322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 632/1263] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:01.477528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.477871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.478071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.478261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.478946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.481377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.487882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a01f23-f682-408b-a04b-fceea4c31d1a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.492382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.493035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.493309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.493574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] PATH_INFO: `/v3/projects/3c9d4b857e654cb38c82b0a28c5ccd77/users/37a30e3348be4d02b8b827b7eb32ba6a/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.507481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 632/1264] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.507481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.508826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9d4b857e654cb38c82b0a28c5ccd77, user_id=37a30e3348be4d02b8b827b7eb32ba6a, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.511574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcab108-327a-4040-bb7f-eb50480a8939 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.517005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8e08-da28-4366-8837-eca4fa445fe9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.522044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.522639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.522911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.523208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.523868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.525740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.526287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 633/1265] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:01 2022] PUT /identity/v3/projects/3c9d4b857e654cb38c82b0a28c5ccd77/users/37a30e3348be4d02b8b827b7eb32ba6a/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:01.527532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.527816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.528086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.553662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 633/1266] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/users => generated 285 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.553662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.561488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f75a1-795f-498a-99fd-6308fc6f1cb0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.562189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] MFA Rules not processed for user `68e23f85e2a244b8bddf47beb3da5da8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:01.563545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:01.626173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.626173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.626173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.626173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.626708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.626708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.632160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 634/1267] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:01.636500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.637163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 634/1268] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.638012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.641660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.648196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315d0a-59fa-4ca8-a15e-62d272dcf334 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.649575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb7954-81ae-4828-a13a-547123e10ec5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.653133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a132107-0834-472d-944b-84614941684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.653728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a132107-0834-472d-944b-84614941684a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.654008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a132107-0834-472d-944b-84614941684a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.654284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a132107-0834-472d-944b-84614941684a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.654605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.654920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a132107-0834-472d-944b-84614941684a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.655263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.655566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.655840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.656543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.656647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a132107-0834-472d-944b-84614941684a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.658284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.668315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 635/1269] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.669007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 635/1270] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.669612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a132107-0834-472d-944b-84614941684a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.671994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.679133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a132107-0834-472d-944b-84614941684a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.684008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.684708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.685046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.685318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.686545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.688263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eabe16-6cd7-4ac1-a4bf-fcfcd3e4093c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.698131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 636/1271] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.699321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.699605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.699876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.719176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:01.720370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:01.726148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2753-95c9-475a-a972-bc413cd4ab17 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.727763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 637/1272] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:01.728513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.733282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.734302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.734302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.734519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.737632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265215f-1c64-4503-9e8d-2e10ec28046d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.742460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.743049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.743350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.743621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.744257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.744659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.745895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.746230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.755794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 638/1273] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.756259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 636/1274] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:01.756553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.757038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.762531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:01.767726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930575b9-6e4d-4575-a046-d64c57d913dd service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.769415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51734-4fb5-43b6-9c80-3b693bc071d5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.774096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.774242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.774513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.774781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.775456 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.776510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.777043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.777442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.777722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.778332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.779512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.782147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.791680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 637/1275] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.791680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.801979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34802f3-ec62-4080-a95b-7a4ecdc83a0b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.807141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.807779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.808246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.808680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.809416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.811396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.820774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 639/1276] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:01 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:01.820774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.826898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 638/1277] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.826898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.836588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801d00-83de-4925-9ac5-06bc80e3925c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.896157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66aa6-6e73-4e11-91d8-434c0f4c2e37 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.901810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.902546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.902826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.903200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.904039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.905647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.906790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.907356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.907636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.907911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.908693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.910328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.921386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 640/1278] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.921386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.921637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 639/1279] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.922167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.935358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c780496-b73a-449b-9ea7-210eeac24d18 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.938088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.938696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.938970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.939288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.939936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.941542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:01.957111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 640/1280] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:01.958973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:01.971431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58747e51-93a0-42e8-a00b-28b892114045 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.976582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:01.977365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:01.977746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:01.978120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] PATH_INFO: `/v3/projects/833e12acbbd34fc6b666cd3d6e9fc9e2/users/fdca15e88c7b422584f3d0a95270ad5d/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:01.994232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833e12acbbd34fc6b666cd3d6e9fc9e2, user_id=fdca15e88c7b422584f3d0a95270ad5d, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:01.997319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742061-5535-47ff-be91-0f0670ea636f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:01.997512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.001302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.001911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.002365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.002494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.022324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 641/1281] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:01 2022] PUT /identity/v3/projects/833e12acbbd34fc6b666cd3d6e9fc9e2/users/fdca15e88c7b422584f3d0a95270ad5d/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.022324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.022655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.022655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.025499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 641/1282] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:02.026291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.036181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aaa26-77f1-4305-ba3f-780b65e45042 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.074737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:02.109965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70875d0-3329-4917-aae2-1714f991cee9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.114901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.115690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.116247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.116878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.117713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.120157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.121960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.123117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.123600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.124080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] PATH_INFO: `/v3/projects/230f3c911ac84f589b9b08b7d910e34d/users/28e088c7bf2f41a397d41e8d3c2d7bae/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.140980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230f3c911ac84f589b9b08b7d910e34d, user_id=28e088c7bf2f41a397d41e8d3c2d7bae, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.144295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.152658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 642/1283] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:02.154342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.169552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ead49-1544-4234-b973-198c26ef9a51 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.176381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 642/1284] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:02 2022] PUT /identity/v3/projects/230f3c911ac84f589b9b08b7d910e34d/users/28e088c7bf2f41a397d41e8d3c2d7bae/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.178450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.188755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:02.259246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.260772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.261188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.261550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] PATH_INFO: `/v3/projects/807408a17e8e4e8786b2f906a126c3d1/users/6201328900324facb4c2a1c118023b73/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.266880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ed704-c77c-49bf-8460-ce791e20c872 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.273764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.274845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.275563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.276018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] PATH_INFO: `/v3/projects/0be51f2486df4f52861e160ebbe70928/users/731b38c56a474a70ac8bcfdf641e98a6/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.284093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807408a17e8e4e8786b2f906a126c3d1, user_id=6201328900324facb4c2a1c118023b73, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.287064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.309321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be51f2486df4f52861e160ebbe70928, user_id=731b38c56a474a70ac8bcfdf641e98a6, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.314685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7bba0-30f9-4356-82df-b60aa3b81747 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.327038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 643/1285] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:02 2022] PUT /identity/v3/projects/807408a17e8e4e8786b2f906a126c3d1/users/6201328900324facb4c2a1c118023b73/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.328193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.350535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 643/1286] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:02 2022] PUT /identity/v3/projects/0be51f2486df4f52861e160ebbe70928/users/731b38c56a474a70ac8bcfdf641e98a6/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.350535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.350535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.350535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.382296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] MFA Rules not processed for user `fdca15e88c7b422584f3d0a95270ad5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:02.384142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:02.417991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd487b4-d4e6-4b56-b050-45e4d5ed2263 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.475609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 644/1287] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:02.481398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.497217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:02.523866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.524567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.524851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.525129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] PATH_INFO: `/v3/projects/c3357decaf034d71a3b93494594f18e1/users/9a299bf3011d48c5ae0ba69f18ca8847/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.551473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3357decaf034d71a3b93494594f18e1, user_id=9a299bf3011d48c5ae0ba69f18ca8847, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.556898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a13e31-f626-4cc2-949d-28e930d6db3f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.569349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64454c-14d5-4496-a183-a6f310d67920 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.578028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 644/1288] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:02 2022] PUT /identity/v3/projects/c3357decaf034d71a3b93494594f18e1/users/9a299bf3011d48c5ae0ba69f18ca8847/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.580306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0843-1da8-4441-ad63-e8accfe9057b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.580887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0843-1da8-4441-ad63-e8accfe9057b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.581172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f0843-1da8-4441-ad63-e8accfe9057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.593061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:02.608837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6e1f0843-1da8-4441-ad63-e8accfe9057b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:02.610437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6e1f0843-1da8-4441-ad63-e8accfe9057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:02.671620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.672983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.673843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.674475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.675642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.679166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae64a36-df0c-4060-bca6-d5040def970e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.695216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 645/1289] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:02.697113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.697468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.697805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.697926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 645/1290] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:02.700449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3d432-4a9d-4baf-b003-fff687b531b0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.700832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3d432-4a9d-4baf-b003-fff687b531b0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.701183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3d432-4a9d-4baf-b003-fff687b531b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.727605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b3e3d432-4a9d-4baf-b003-fff687b531b0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:02.729285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b3e3d432-4a9d-4baf-b003-fff687b531b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:02.737135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 646/1291] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:02.737339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] MFA Rules not processed for user `6201328900324facb4c2a1c118023b73`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:02.738427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94346-502a-48fe-ae48-881c21ffbe13 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.738707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94346-502a-48fe-ae48-881c21ffbe13 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.738985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94346-502a-48fe-ae48-881c21ffbe13 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.739383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:02.743841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 647/1292] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:24:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:02.746228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457d5f-ff0d-4a7b-bcff-78283b540207 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.746582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457d5f-ff0d-4a7b-bcff-78283b540207 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.746924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457d5f-ff0d-4a7b-bcff-78283b540207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.812572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d8457d5f-ff0d-4a7b-bcff-78283b540207 None None] MFA Rules not processed for user `9a299bf3011d48c5ae0ba69f18ca8847`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:02.813807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d8457d5f-ff0d-4a7b-bcff-78283b540207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:02.878303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 646/1293] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:02.879410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.894094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf3c83-07d9-4860-831a-2f52ba7c8fe1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:02.905355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:02.906530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.906957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.907347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] PATH_INFO: `/v3/projects/4e6399d269da4d39b252188bcdc7f5a0/users/958a03baeb494ffa8f5e9c02b779d651/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:02.931660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6399d269da4d39b252188bcdc7f5a0, user_id=958a03baeb494ffa8f5e9c02b779d651, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:02.935275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:02.954360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 647/1294] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:02 2022] PUT /identity/v3/projects/4e6399d269da4d39b252188bcdc7f5a0/users/958a03baeb494ffa8f5e9c02b779d651/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:02.955480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:02.974212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 648/1295] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:02.975553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:02.975840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:02.976099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.046815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:03.048150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:03.101028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3db4c-ed3e-42d6-bc02-447ef54e17b9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.112287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.113287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.116163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.116613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.122533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:03.186437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 649/1296] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:03.187183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.201117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:03.209953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9663bf-9f61-481a-a14c-3e775e1012b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.220999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.222261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.223307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.224790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.242528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.246358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.247822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.248840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.272619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 648/1297] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:03.273605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.275060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 650/1298] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:03.276159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.293910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79145e-e305-4a5a-adfc-7297f9983e6a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.314138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.314405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.314596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.314926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.315755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.317921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc835c-51e8-4514-bc7c-b4f4d1f4b39f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.340010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 651/1299] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:03.341362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc44753-6660-477a-a0f2-b1284e9d3a00 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.341694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc44753-6660-477a-a0f2-b1284e9d3a00 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.341978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc44753-6660-477a-a0f2-b1284e9d3a00 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.343713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 652/1300] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:03.344429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc44753-6660-477a-a0f2-b1284e9d3a00 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.351994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d182f1-5a04-4868-a069-6d5f9387519d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.360903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.362269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc44753-6660-477a-a0f2-b1284e9d3a00 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.362405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.363014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.365241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.369699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.371096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.371485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.371933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.477684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.479851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1f602-f169-47d8-a2a7-5522644e598c service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.480045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.482151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.494809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 649/1301] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:03.496266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.502910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 653/1302] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:03.506871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.507312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.508608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.512862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaafa94-c691-49e2-8933-eff8c1ab99a5 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.519172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.520304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.521936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.522426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] PATH_INFO: `/v3/projects/4e6399d269da4d39b252188bcdc7f5a0/users/958a03baeb494ffa8f5e9c02b779d651/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.525199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6399d269da4d39b252188bcdc7f5a0, user_id=958a03baeb494ffa8f5e9c02b779d651, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.530823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e74e9-e7dd-4030-9b11-002ec1c2d045 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.547436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:03.549004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:03.569370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 650/1303] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:03 2022] PUT /identity/v3/projects/4e6399d269da4d39b252188bcdc7f5a0/users/958a03baeb494ffa8f5e9c02b779d651/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:03.572518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.572824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.573472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.648458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] MFA Rules not processed for user `958a03baeb494ffa8f5e9c02b779d651`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:03.651740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:03.709422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 654/1304] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:24:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:03.710901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.732193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a0e1d-75c6-4a43-a7f2-bb441b7ff6f7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:03.747764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:03.748370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:03.748650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:03.748918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:03.839524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:03.858338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 651/1305] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:03.867906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:03.872504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:03.880056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:03.881708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929f610-6b7d-4c32-bb9c-16c6fe300880 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:03.901424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 655/1306] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:03.993778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a901c1-8ade-481d-9c6f-e1462a0cda82 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.006911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.007589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.007868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.008138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.028139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:04.029902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66a04d-da2b-47a0-b73e-c857eeb4ccd6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:04.056756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 652/1307] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:04.119871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f85-43da-4fa6-98fd-5b030de0cfa3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.120509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f85-43da-4fa6-98fd-5b030de0cfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.120509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f87f85-43da-4fa6-98fd-5b030de0cfa3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.121743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 656/1308] 10.222.0.36 () {54 vars in 912 bytes} [Tue Oct 25 13:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:04.147373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1c50-cc98-4b8c-877b-4045d8ad9563 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.147373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1c50-cc98-4b8c-877b-4045d8ad9563 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.147373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe1c50-cc98-4b8c-877b-4045d8ad9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.173466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-05fe1c50-cc98-4b8c-877b-4045d8ad9563 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:04.174753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05fe1c50-cc98-4b8c-877b-4045d8ad9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:04.200674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 653/1309] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:24:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:04.226237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87f85-43da-4fa6-98fd-5b030de0cfa3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:04.251924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f87f85-43da-4fa6-98fd-5b030de0cfa3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.256011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.256639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.256928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.257207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.278651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:04.280145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61dec5-7abc-4e5e-95d1-306498b0bccd service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:04.300461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 657/1310] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:04.385274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef079c1c-778e-4c18-8114-d9fba39d4f5b None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.386251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef079c1c-778e-4c18-8114-d9fba39d4f5b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.386767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef079c1c-778e-4c18-8114-d9fba39d4f5b None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.389512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 654/1311] 10.222.0.36 () {54 vars in 912 bytes} [Tue Oct 25 13:24:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:04.396454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.397063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.397566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.440044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:04.441602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:04.467559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 658/1312] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:24:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 25 13:24:04.500685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef079c1c-778e-4c18-8114-d9fba39d4f5b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:04.519178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef079c1c-778e-4c18-8114-d9fba39d4f5b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.525444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.526812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.527825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.528332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.543836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:04.548259 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:04.568368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 655/1313] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:04.809828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:04.848538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:04.865968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4277f-4ea6-4d35-9087-0da70f6696e5 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.884017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '958a03baeb494ffa8f5e9c02b779d651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e6399d269da4d39b252188bcdc7f5a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.884954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.885802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.885802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.886242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:04.886551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:04.888899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:04.922389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 656/1314] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:24:04.924981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:04.949913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f68c53-0fe3-4fe3-8a84-f66ec4e76c28 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.963085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '958a03baeb494ffa8f5e9c02b779d651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e6399d269da4d39b252188bcdc7f5a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.964910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.965921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.966398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:04.968306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:04.977019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:04.982733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8647b13-9a24-4c1f-8cb4-0bf893bc03f6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:04.994253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:04.994822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:04.995100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:04.997524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] PATH_INFO: `/v3/users/fdca15e88c7b422584f3d0a95270ad5d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:05.001515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdca15e88c7b422584f3d0a95270ad5d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:05.007535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:05.034922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 657/1315] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:04 2022] POST /identity/v3/projects => generated 354 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:05.038833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:05.132689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdca15e88c7b422584f3d0a95270ad5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.171736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdca15e88c7b422584f3d0a95270ad5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.172458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdca15e88c7b422584f3d0a95270ad5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.189149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432c7b-7865-4f9e-9d8a-ca7ce468e1ea tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:05.191394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdca15e88c7b422584f3d0a95270ad5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.193229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2724389d-40db-4d13-8ccc-c91c5f362515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdca15e88c7b422584f3d0a95270ad5d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.205752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '958a03baeb494ffa8f5e9c02b779d651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e6399d269da4d39b252188bcdc7f5a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:05.206920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:05.207994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:05.208293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:05.210521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:05.210863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:05.216950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:05.226120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 659/1316] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:04 2022] DELETE /identity/v3/users/fdca15e88c7b422584f3d0a95270ad5d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:05.230516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:05.231206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:05.232414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:05.248782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 658/1317] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:05.286244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:05.289311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:05.321566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:05.323671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:05.339407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026da1-30dd-4a05-be22-9dc1d1f37ec9 tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:05.381571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:05.430574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:05.458110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 660/1318] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:05.465014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '958a03baeb494ffa8f5e9c02b779d651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e6399d269da4d39b252188bcdc7f5a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:05.465697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:05.465983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:05.466256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] PATH_INFO: `/v3/projects/0ed0644a9812422895962983876cc4d8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:05.486068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0ed0644a9812422895962983876cc4d8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:05.488113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:05.512310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:05.513752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:05.715241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710f078-f802-4146-9d2f-3b1635fd64df None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:05.724554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed0644a9812422895962983876cc4d8', 'request_id': 'req-0c00379c-f43a-47e4-a277-fbc31ec1705c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.749128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed0644a9812422895962983876cc4d8', 'request_id': 'req-0c00379c-f43a-47e4-a277-fbc31ec1705c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.773754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed0644a9812422895962983876cc4d8', 'request_id': 'req-0c00379c-f43a-47e4-a277-fbc31ec1705c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.804611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] The token cache is being invalidate because project 0ed0644a9812422895962983876cc4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:05.804818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.807863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:05.811558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 659/1319] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:05 2022] DELETE /identity/v3/projects/0ed0644a9812422895962983876cc4d8 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:05.962653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:05.963219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:05.963575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:05.963762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:05.974128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:06.164273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:06.166468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f5cda-da05-43eb-8bd8-b406ed28d2e4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:06.186800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 661/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:06.454382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:06.456012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:06.481124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00379c-f43a-47e4-a277-fbc31ec1705c tempest-VolumeQuotaClassesTest-1727235217 tempest-VolumeQuotaClassesTest-1727235217-project-admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:06.496524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:06.497141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:06.497427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:06.497997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] PATH_INFO: `/v3/users/68e23f85e2a244b8bddf47beb3da5da8` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:06.499752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e23f85e2a244b8bddf47beb3da5da8)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:06.500797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: WARNING py.warnings [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:24:06.500797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: warnings.warn(msg) Oct 25 13:24:06.500797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: Oct 25 13:24:06.502024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:06.630950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e23f85e2a244b8bddf47beb3da5da8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:06.670581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e23f85e2a244b8bddf47beb3da5da8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:06.690282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e23f85e2a244b8bddf47beb3da5da8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:06.691752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e23f85e2a244b8bddf47beb3da5da8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:06.693454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e23f85e2a244b8bddf47beb3da5da8'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:06.721797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 660/1321] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:06 2022] DELETE /identity/v3/users/68e23f85e2a244b8bddf47beb3da5da8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:06.726561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918b3f9-d7f7-4e34-8ee6-f34e1bee229c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:06.726858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918b3f9-d7f7-4e34-8ee6-f34e1bee229c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:06.727331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918b3f9-d7f7-4e34-8ee6-f34e1bee229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:06.767726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9918b3f9-d7f7-4e34-8ee6-f34e1bee229c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:06.769201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9918b3f9-d7f7-4e34-8ee6-f34e1bee229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:06.869233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:06.898280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 662/1322] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:06.974846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:06.976814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:06.977653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:06.978186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:06.979080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:07.057485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] MFA Rules not processed for user `731b38c56a474a70ac8bcfdf641e98a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:07.062216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:07.137511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:07.152296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cac60-dc82-4ce2-b4ac-1b4f45a2715b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.165182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.174367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.174612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.176333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.209957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.214533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.259515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 661/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:07.259709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 663/1324] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:07.264598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.271609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:07.292670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3d58-d6a2-44d4-8f82-33833f046f9f service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.297795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.298435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.298718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.299371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.300749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.302269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.358048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 662/1325] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:07 2022] POST /identity/v3/projects => generated 401 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:07.361960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.378006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a98aad-64a7-4296-95d4-032214c28d73 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.516349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.518168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:07.557907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.558549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.559548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.560024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.560860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.563017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.647803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 664/1326] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:07 2022] POST /identity/v3/users => generated 307 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:07.648807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.680231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2987f-e9a1-4561-bf93-03278afcd3c0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.703926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.704835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.706970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.707620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.710497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.714386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.731663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4fa1-6129-4f42-98a9-dbba7e91eb7f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.750352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.751301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.751738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.753062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] PATH_INFO: `/v3/projects/833e12acbbd34fc6b666cd3d6e9fc9e2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.755528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833e12acbbd34fc6b666cd3d6e9fc9e2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.758665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 665/1327] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:07.760997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.774346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.788090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afb9cb-337e-4956-b88b-585b74237717 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:07.801979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:07.804570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.806451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.807444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] PATH_INFO: `/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f/users/1d310e5e3fb14a0ebf01d822779d1c7e/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:07.854419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc177b2c142e4ee1a47db065e46b3a5f, user_id=1d310e5e3fb14a0ebf01d822779d1c7e, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:07.859393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:07.894439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 666/1328] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:07 2022] PUT /identity/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f/users/1d310e5e3fb14a0ebf01d822779d1c7e/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:07.895935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:07.913308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833e12acbbd34fc6b666cd3d6e9fc9e2', 'request_id': 'req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:07.924952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833e12acbbd34fc6b666cd3d6e9fc9e2', 'request_id': 'req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:07.943484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833e12acbbd34fc6b666cd3d6e9fc9e2', 'request_id': 'req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:07.966287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] The token cache is being invalidate because project 833e12acbbd34fc6b666cd3d6e9fc9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:07.966588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:07.967341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b22c78dd-01bc-4a8a-94cb-b798b06e3b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:07.974834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 663/1329] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:07 2022] DELETE /identity/v3/projects/833e12acbbd34fc6b666cd3d6e9fc9e2 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:07.975768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:07.979525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456d45-da30-4041-aa73-6e6968afc42b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:07.980196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456d45-da30-4041-aa73-6e6968afc42b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:07.981383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456d45-da30-4041-aa73-6e6968afc42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.032372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c2456d45-da30-4041-aa73-6e6968afc42b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:08.033191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c94f4-3603-43a7-8534-d67479e19e6d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.044649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c2456d45-da30-4041-aa73-6e6968afc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:08.150906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.153025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.153893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.154367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.155850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.158313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:08.184995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 667/1330] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:08.191511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.195771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:08.226533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:08.238752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761b6b8-2558-4ee0-853d-9dfd4f93d11d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.257879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 664/1331] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:08.260128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.260311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.261031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.261309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.261578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] PATH_INFO: `/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f/users/1d310e5e3fb14a0ebf01d822779d1c7e/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.261819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.262254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.264709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc177b2c142e4ee1a47db065e46b3a5f, user_id=1d310e5e3fb14a0ebf01d822779d1c7e, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.267964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:08.306117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:08.307690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:08.313423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 668/1332] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:08 2022] PUT /identity/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f/users/1d310e5e3fb14a0ebf01d822779d1c7e/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:08.453409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.454737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 665/1333] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:08.481697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229b2e2-ec0f-471c-a385-16d0ff427683 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.493782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.493782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.493782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.493782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.495575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:08.495575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.495575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:08.513962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.516393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:08.540146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 669/1334] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:24:08.544774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.550150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906985ef-c0e9-46ea-a22d-036e2b1bb635 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.558215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.559513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.560077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.560519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] PATH_INFO: `/v3/users/9a299bf3011d48c5ae0ba69f18ca8847` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.562056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a299bf3011d48c5ae0ba69f18ca8847)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.564986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:08.583213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ae5e-cbe1-4660-aa67-1ef989c62ac2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.608101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.608885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.609169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.609436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.610092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.621434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:08.688974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 670/1335] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:08 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:08.692737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.700821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a299bf3011d48c5ae0ba69f18ca8847'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:08.739613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a299bf3011d48c5ae0ba69f18ca8847'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:08.749427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a299bf3011d48c5ae0ba69f18ca8847'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:08.750445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a299bf3011d48c5ae0ba69f18ca8847'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:08.751346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a299bf3011d48c5ae0ba69f18ca8847'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:08.788232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 666/1336] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:08 2022] DELETE /identity/v3/users/9a299bf3011d48c5ae0ba69f18ca8847 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:08.788232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:08.792298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:08.836209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ac8d0-5382-4be1-b2ce-dc5b129029f9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:08.907276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:08.958113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:08.958751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:08.959023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:08.959317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:08.959973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:08.961925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad58e98-daf1-4647-bd6e-1cb61da08341 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.003258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310d106-0371-4b14-b07a-92fa213399d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.011140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.012039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.012735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 671/1337] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:08 2022] POST /identity/v3/users => generated 291 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:09.014090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.014371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.014644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.016493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.017051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] PATH_INFO: `/v3/projects/832915e9a8694e129b533ee5f00cffc2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.018746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832915e9a8694e129b533ee5f00cffc2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.021174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.058831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:09.060124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.063304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:09.077576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 672/1338] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:09.080773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:09.082791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.104886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d908104-d48c-4ba3-a12e-a2a037bdec61 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.117886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.118527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.118808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.119074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.119779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.121510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.148984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 673/1339] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:09.150312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:09.157722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.222898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832915e9a8694e129b533ee5f00cffc2', 'request_id': 'req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.248094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832915e9a8694e129b533ee5f00cffc2', 'request_id': 'req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.269317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832915e9a8694e129b533ee5f00cffc2', 'request_id': 'req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.299473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] The token cache is being invalidate because project 832915e9a8694e129b533ee5f00cffc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:09.299913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.301361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.314439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 667/1340] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:08 2022] DELETE /identity/v3/projects/832915e9a8694e129b533ee5f00cffc2 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:09.316105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:09.333079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.403075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cc48e-8275-4b53-b0bf-488afb36cc67 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.604292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e94fe-2ae1-4876-9a3b-3b7f0756e7c2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.605478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.606970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.607436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.607857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.611837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.621652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.623130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.623618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.624742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.627070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.628913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.651578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.654494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.659114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 668/1341] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:09.662681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:09.672044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.682136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 674/1342] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:09.682864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:09.702044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609759-f524-449b-827c-acb7650cd45e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.716903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.717583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.717859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.718130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] PATH_INFO: `/v3/users/958a03baeb494ffa8f5e9c02b779d651` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.720370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=958a03baeb494ffa8f5e9c02b779d651)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.722364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.726377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c6413-423d-4065-85e2-c61b325e1005 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:09.736545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:09.737817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.739528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.739818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] PATH_INFO: `/v3/projects/b1637e3f23174ca1803584e65bef1843/users/054d87c1c9f4444184c8b00b5e3d0dd9/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.784164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1637e3f23174ca1803584e65bef1843, user_id=054d87c1c9f4444184c8b00b5e3d0dd9, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:09.792806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce58f4-e122-4fa4-9037-d2938d40b432 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:09.839659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 675/1343] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:09 2022] PUT /identity/v3/projects/b1637e3f23174ca1803584e65bef1843/users/054d87c1c9f4444184c8b00b5e3d0dd9/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:09.841581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e894c-2041-417b-b2de-1626853c701f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:09.842369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e894c-2041-417b-b2de-1626853c701f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:09.842838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e894c-2041-417b-b2de-1626853c701f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:09.843947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958a03baeb494ffa8f5e9c02b779d651'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.867466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958a03baeb494ffa8f5e9c02b779d651'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.884093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958a03baeb494ffa8f5e9c02b779d651'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.884562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-382e894c-2041-417b-b2de-1626853c701f None None] MFA Rules not processed for user `054d87c1c9f4444184c8b00b5e3d0dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:09.885573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958a03baeb494ffa8f5e9c02b779d651'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.885979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-382e894c-2041-417b-b2de-1626853c701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:09.888094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958a03baeb494ffa8f5e9c02b779d651'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:09.924003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 669/1344] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:09 2022] DELETE /identity/v3/users/958a03baeb494ffa8f5e9c02b779d651 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:09.949779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:10.056527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 676/1345] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:10.056760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:10.058197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:10.069895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:10.164432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82023660-ef61-4096-b437-e6266d9532a8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:10.177977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:10.178788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:10.179247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:10.179628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:10.180455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:10.182639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:10.220760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 670/1346] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:10 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:10.222528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e894c-2041-417b-b2de-1626853c701f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:10.395802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e894c-2041-417b-b2de-1626853c701f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:10.590871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:10.591670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:10.592157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:10.592504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:10.593225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:10.596934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:10.705500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 677/1347] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:10 2022] POST /identity/v3/users => generated 296 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:10.710080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:10.734974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12741b9b-9f5b-4458-962d-077306c2fb4f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:10.747663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:10.749108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:10.749392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:10.751102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:10.756090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:10.758324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:10.801616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 671/1348] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:10.809705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:10.855632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444250-3a7d-47a8-9206-96b6fbb5060f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:10.862458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:10.863427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:10.863885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:10.864303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] PATH_INFO: `/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54/users/68f6f3ec94c5423c8f64c17870dd5c47/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:10.892894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ff0ee0bbd044acb4ef1c7f8877bb54, user_id=68f6f3ec94c5423c8f64c17870dd5c47, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:10.900216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:10.921137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 678/1349] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:10 2022] PUT /identity/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54/users/68f6f3ec94c5423c8f64c17870dd5c47/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:10.924879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:10.942598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b79d-489f-4ed9-a6eb-ad0d25d01a91 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.105322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.106159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.106449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.106724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.107517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.109649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:11.126219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 672/1350] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:11.130220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.158484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.161577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489c1ce-e8f5-4b01-a9de-1d1e7093a5a0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.163633 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:11.185991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.189528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.190630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.191106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] PATH_INFO: `/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54/users/68f6f3ec94c5423c8f64c17870dd5c47/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.197959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ff0ee0bbd044acb4ef1c7f8877bb54, user_id=68f6f3ec94c5423c8f64c17870dd5c47, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.202077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080cd34-dbb4-4fd9-ac4a-1acc27f53d1b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:11.207172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3805d18-d020-4cab-91f0-960cf75e35fc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.225151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.232004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.232795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.233217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] PATH_INFO: `/v3/projects/c3357decaf034d71a3b93494594f18e1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.235343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3357decaf034d71a3b93494594f18e1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.237204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 679/1351] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:11 2022] PUT /identity/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54/users/68f6f3ec94c5423c8f64c17870dd5c47/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:11.239463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:11.248290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.248574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.248854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.314939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] MFA Rules not processed for user `68f6f3ec94c5423c8f64c17870dd5c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:11.316423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:11.339105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:11.360917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3357decaf034d71a3b93494594f18e1', 'request_id': 'req-0178a0c5-0018-419d-8275-826dec7bf901'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:11.382162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3357decaf034d71a3b93494594f18e1', 'request_id': 'req-0178a0c5-0018-419d-8275-826dec7bf901'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:11.398724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3357decaf034d71a3b93494594f18e1', 'request_id': 'req-0178a0c5-0018-419d-8275-826dec7bf901'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:11.421835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] The token cache is being invalidate because project c3357decaf034d71a3b93494594f18e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:11.422129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:11.423614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0178a0c5-0018-419d-8275-826dec7bf901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:11.429832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 673/1352] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:11 2022] DELETE /identity/v3/projects/c3357decaf034d71a3b93494594f18e1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:11.431870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749ef6-a3ab-41d5-a824-928607354840 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.432151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749ef6-a3ab-41d5-a824-928607354840 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.432425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749ef6-a3ab-41d5-a824-928607354840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.451116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:11.461330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 680/1353] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:11.476506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.478197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:11.497337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8c749ef6-a3ab-41d5-a824-928607354840 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:11.498649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8c749ef6-a3ab-41d5-a824-928607354840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:11.635050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8d80-e5f3-459a-902e-dee2ead12cae None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.643284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 674/1354] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:11.652584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.653532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.654040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.708510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:11.712732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:11.734877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 675/1355] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:11.738860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.752810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46760343-9d23-4d99-839b-ddaf345af63b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.768464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.769329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.769785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.770144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.771970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.772614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.772887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.773157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.773817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:11.774114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.775754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:11.809973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 676/1356] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:24:11.814329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.838502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef19cc-1e60-473d-9820-39fa09b8f623 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:11.870036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:11.870763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:11.871042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:11.871351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:11.872001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.880754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:11.978524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 677/1357] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:11 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:11.982137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:11.982426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:11.984256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.011503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 681/1358] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:12.180167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd552b-7ad6-495c-a137-94e751d613a7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.195474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.199564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.200061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.200484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.202237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:12.205697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.273154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:12.287929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f345c7-1900-4b7f-babd-008e157eb183 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.307537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 678/1359] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:11 2022] POST /identity/v3/users => generated 282 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:12.310855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:12.336456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ff85-f10f-4be4-86bb-a66355b27173 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.344725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:12.350607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.351658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.355117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.357511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.358460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:12.362107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.399455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 679/1360] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:12.404417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:12.427119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e64-8286-44c7-8172-6fb792d5bddd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.433906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.434673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.434962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.435308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.436153 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:12.439717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.467073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 680/1361] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:12.467492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.468073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:12.468584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.470283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:12.470547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.471026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.476527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:12.494336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b816413-ea56-48ce-9790-1f99a1c9c456 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.514851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.523392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.523684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.523849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] PATH_INFO: `/v3/projects/4e6399d269da4d39b252188bcdc7f5a0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.527464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6399d269da4d39b252188bcdc7f5a0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:12.534150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.633242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:12.678629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:12.680831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:12.696416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6399d269da4d39b252188bcdc7f5a0', 'request_id': 'req-36614607-77fc-4bac-9ea4-89606c2c5fb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:12.718283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 682/1362] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:12.719101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:12.741257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6399d269da4d39b252188bcdc7f5a0', 'request_id': 'req-36614607-77fc-4bac-9ea4-89606c2c5fb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:12.763133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6399d269da4d39b252188bcdc7f5a0', 'request_id': 'req-36614607-77fc-4bac-9ea4-89606c2c5fb9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:12.771092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0552e0-f631-4f42-8f5b-55af5e756fde service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:12.784667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] The token cache is being invalidate because project 4e6399d269da4d39b252188bcdc7f5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:12.785215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:12.787089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36614607-77fc-4bac-9ea4-89606c2c5fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:12.790427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 681/1363] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:12 2022] DELETE /identity/v3/projects/4e6399d269da4d39b252188bcdc7f5a0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:12.795682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cd1-13c1-46c4-bb30-53b9779fbeb6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.796214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cd1-13c1-46c4-bb30-53b9779fbeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.796635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cd1-13c1-46c4-bb30-53b9779fbeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:12.856069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b7a65cd1-13c1-46c4-bb30-53b9779fbeb6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:12.857327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b7a65cd1-13c1-46c4-bb30-53b9779fbeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:12.975108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:12.976164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:12.976597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:12.977045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] PATH_INFO: `/v3/projects/c905bb2151134069b7eb82242f816236/users/da49ebb09988435d96f27fdaca42918b/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.013408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c905bb2151134069b7eb82242f816236, user_id=da49ebb09988435d96f27fdaca42918b, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.025774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c0077-0dee-4ced-a284-221ca3c7553e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.035313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 682/1364] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:13.045226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.045534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.045784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.073970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 683/1365] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:12 2022] PUT /identity/v3/projects/c905bb2151134069b7eb82242f816236/users/da49ebb09988435d96f27fdaca42918b/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:13.087315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:13.088423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.088593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.089013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.090660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:13.147281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] MFA Rules not processed for user `da49ebb09988435d96f27fdaca42918b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:13.148554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:13.293668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:13.302278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 683/1366] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:13.305958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.326380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c0592-7db7-452d-bd03-a83bed779642 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.346485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.346485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.346485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.346485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.347324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:13.348242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.350050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.362251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 684/1367] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:13.367920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.370867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:13.395727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 684/1368] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:13.402743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.406996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630310c2-13e7-47e8-8c18-adcc6ff8c40c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.420746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.421533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.422055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.422704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.423780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.429325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.433231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167efa72-9b3d-47e1-9047-0ae673da1e99 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.453234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.453803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.454171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.454481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.455139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.457704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.496553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 685/1369] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:13.500769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.531168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 685/1370] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:13.533333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.591858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:13.674857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2e523-77ae-4bb5-8eeb-107e547e0539 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.729679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ed0a8-e0cc-429e-977c-a04afbaa2f25 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.740103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.741099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.741733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.743579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.744367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.748481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d713f-2280-4412-870b-1a60ad449641 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.812717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 686/1371] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/users => generated 284 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:13.814850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e732da-4a72-4bde-94c1-cf3d75620a13 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.816392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e732da-4a72-4bde-94c1-cf3d75620a13 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.818151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e732da-4a72-4bde-94c1-cf3d75620a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.844851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.846283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.846747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.847219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.848111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.852134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.888561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d7e732da-4a72-4bde-94c1-cf3d75620a13 None None] MFA Rules not processed for user `1d310e5e3fb14a0ebf01d822779d1c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:13.895450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7e732da-4a72-4bde-94c1-cf3d75620a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:13.900037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 686/1372] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/users => generated 288 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:13.900037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.924767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e1c27-a7d6-4860-a6e5-8e7f4400a9c0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.936309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.936956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.937234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.937506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.938166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.939831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:13.967847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 687/1373] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:13.969556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:13.989167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44a337-382f-4608-a399-025fffec3441 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:13.994538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:13.995518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:13.996053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:13.996393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:13.997232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:13.999984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.015854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 687/1374] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:13 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:14.015854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.015854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.015854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.016305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 688/1375] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:14.017378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.030070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac53eb-dac7-47c5-9e82-dfd7b95f1cb4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.036714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.043639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.044021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.044472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.045245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:14.047225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.056461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] MFA Rules not processed for user `213a6f956e5c48b4b45a6481e29db89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:14.057923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:14.060426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 689/1376] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:14.061240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.074300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd3df-44bd-45d9-9f93-843cb329030f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.079964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.080762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.081163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.082449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] PATH_INFO: `/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41/users/5a8db1c37ff841118b479e52cef4b13e/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.103424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f2aeebabaf41369e3c6ad40a5f4b41, user_id=5a8db1c37ff841118b479e52cef4b13e, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:14.106518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.132779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 690/1377] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:14 2022] PUT /identity/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41/users/5a8db1c37ff841118b479e52cef4b13e/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:14.133966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.158924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:14.175984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 688/1378] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:14.186521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.199770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526c180-0a3d-4e16-a238-e46f46949e33 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.257490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:14.343957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.345027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.345161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88b787-d6bb-428a-b389-a6b4f4097967 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.345570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.345986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.347050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:14.351348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb55e5-c114-401d-ae19-fcbe62dfc2c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.354438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.355586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.355933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.356275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] PATH_INFO: `/v3/projects/711e6c16de3f4930857f28814947f041/users/0014081382b241e7b6ba9ea349b41c7d/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.377644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 689/1379] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:14.380598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a385-a7be-4045-a5a9-6927ed668018 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.387304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a385-a7be-4045-a5a9-6927ed668018 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.387655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083a385-a7be-4045-a5a9-6927ed668018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.412972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=711e6c16de3f4930857f28814947f041, user_id=0014081382b241e7b6ba9ea349b41c7d, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:14.424290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.442978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d083a385-a7be-4045-a5a9-6927ed668018 None None] MFA Rules not processed for user `213a6f956e5c48b4b45a6481e29db89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:14.447833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d083a385-a7be-4045-a5a9-6927ed668018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:14.469379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 691/1380] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:14 2022] PUT /identity/v3/projects/711e6c16de3f4930857f28814947f041/users/0014081382b241e7b6ba9ea349b41c7d/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:14.471892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.609542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 690/1381] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:14.615740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32155b-ca80-4519-a7b8-7bf750c0e60c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.615740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32155b-ca80-4519-a7b8-7bf750c0e60c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.615740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32155b-ca80-4519-a7b8-7bf750c0e60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.698395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9d32155b-ca80-4519-a7b8-7bf750c0e60c None None] MFA Rules not processed for user `0014081382b241e7b6ba9ea349b41c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:14.700341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d32155b-ca80-4519-a7b8-7bf750c0e60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:14.700536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322b842-37f2-4228-8ece-eedcf4c98823 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.719140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.719402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.719545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.719852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] PATH_INFO: `/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41/users/5a8db1c37ff841118b479e52cef4b13e/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.723058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f2aeebabaf41369e3c6ad40a5f4b41, user_id=5a8db1c37ff841118b479e52cef4b13e, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:14.730554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:14.760768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 692/1382] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:14 2022] PUT /identity/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41/users/5a8db1c37ff841118b479e52cef4b13e/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:14.763870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:14.766831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:14.848277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 691/1383] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:14.853876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dcb7-ec41-4291-9fb0-a8281be1434c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.854846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dcb7-ec41-4291-9fb0-a8281be1434c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.854983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6dcb7-ec41-4291-9fb0-a8281be1434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:14.940645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e5d6dcb7-ec41-4291-9fb0-a8281be1434c None None] MFA Rules not processed for user `5a8db1c37ff841118b479e52cef4b13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:14.956277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e5d6dcb7-ec41-4291-9fb0-a8281be1434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:14.966454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d53d94-7eda-45d7-937a-3764bab33d18 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:14.986797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:14.987477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:14.987745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:14.988311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.029051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:15.136147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:15.138573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:15.154077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:15.160511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 693/1384] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:15.161443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:15.255629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 692/1385] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:15.257779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.258412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.258894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.308722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8fd65-96b5-459c-9d0b-cdb4cd7aedd8 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:15.320098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:15.320745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.321023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.321295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.321657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] MFA Rules not processed for user `37a30e3348be4d02b8b827b7eb32ba6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:15.321945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:15.323126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:15.328761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:15.414804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 694/1386] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:15 2022] POST /identity/v3/projects => generated 367 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:15.420370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:15.427554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:15.455314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 693/1387] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:15.456255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:15.458664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:15.477571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717069b-e95d-4f91-a9f1-d22bb785c101 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:15.584256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65d83a-44a1-4fc9-95a5-9368016382be admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:15.594877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:15.595619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.595929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.596174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.608467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:15.609408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.609904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.610349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] PATH_INFO: `/v3/users/6201328900324facb4c2a1c118023b73` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.613571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6201328900324facb4c2a1c118023b73)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:15.620190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:15.730953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:15.735643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:15.739879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:15.748721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6201328900324facb4c2a1c118023b73'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:15.760948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 695/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:15.762204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:15.778078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd563f-77eb-4b93-98db-b5aff8badd72 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:15.784782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:15.786071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.786640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.787111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.788017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:15.790485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1b59-3cfa-4fa3-b6f8-cf4c67d900ab admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:15.790781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6201328900324facb4c2a1c118023b73'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:15.801226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:15.805681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6201328900324facb4c2a1c118023b73'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:15.807559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6201328900324facb4c2a1c118023b73'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:15.808463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-09adf187-d6ec-4749-a221-b85410fd5c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6201328900324facb4c2a1c118023b73'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:15.834700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 696/1389] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:15 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:15.837674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350b66-25a9-4e69-9679-3fb96934351a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.838345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350b66-25a9-4e69-9679-3fb96934351a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.838787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350b66-25a9-4e69-9679-3fb96934351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.839843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 694/1390] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:15 2022] DELETE /identity/v3/users/6201328900324facb4c2a1c118023b73 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:15.841676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.841965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.842242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:15.843634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 695/1391] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:15.844335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:15.852262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:15.905959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-84350b66-25a9-4e69-9679-3fb96934351a None None] MFA Rules not processed for user `37a30e3348be4d02b8b827b7eb32ba6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:15.912633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-84350b66-25a9-4e69-9679-3fb96934351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:15.988190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6f863-d3e4-4470-ade7-35e043fe4d57 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:15.997435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:15.998270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:15.999063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:15.999063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.000363 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:16.002901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f26686-9cbc-420d-8e47-6f32e9722b38 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:16.019850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 696/1392] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:16.021558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.021950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.022619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.048208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:16.049874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.056685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 697/1393] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:16.058225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.058514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.058803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.064801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 697/1394] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:16.064801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:16.072473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.111192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:16.115057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.266801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9226e5d-3c71-4e26-90ed-924dbedb9582 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:16.277510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:16.278158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.278430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.278706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.282850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 698/1395] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:24:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:16.283921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.285778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:16.306176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7f266-c5c7-415e-8d05-251ec45e7dcd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:16.320534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:16.321240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.321520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.322441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] PATH_INFO: `/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6/users/48fac9275ffa4a0a8b73800dce4b54a6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.372196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd10e35a6cf4163a7d83cf802c485c6, user_id=48fac9275ffa4a0a8b73800dce4b54a6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:16.375351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:16.403450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 699/1396] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:16 2022] PUT /identity/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6/users/48fac9275ffa4a0a8b73800dce4b54a6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:16.404547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:16.419190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:16.419190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:16.422422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.437252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 698/1397] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:16.438221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:16.439708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.509076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:16.564117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:16.609690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f226a4d-8d5b-4867-9819-1ba5268f10d2 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:16.715193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1d59d-2a92-439a-9c6f-8cccfde76492 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:16.744623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:16.745401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.745678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.745949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.749430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:16.788000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:16.790091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:16.790689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:16.790836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:16.949477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:16.951851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:16.990311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 700/1398] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:16.992525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:16.995969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.006378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.008187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.034191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 699/1399] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:17.038973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.065628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e316a3d-bc55-46e0-8e33-e1a3055f968d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.080991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.081999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.082274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.084694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.086892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.089013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.108889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 700/1400] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:17.109947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.112829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.138041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506ec2a-ebfe-430d-a6c4-6c01851df229 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.158840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.166413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.166413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.166413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] PATH_INFO: `/v3/users/731b38c56a474a70ac8bcfdf641e98a6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.166413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=731b38c56a474a70ac8bcfdf641e98a6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.168801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.237391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fe71-0405-45cd-9c88-078dadd898d0 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.251848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.252558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.252929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.253317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.267818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.273896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207039c6-6e6d-40e9-b1f1-cf392f906033 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.284506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731b38c56a474a70ac8bcfdf641e98a6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:17.290122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 701/1401] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:17.303196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a5b2b-17c7-4892-8287-749bd84a04c2 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.304740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a5b2b-17c7-4892-8287-749bd84a04c2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.305151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a5b2b-17c7-4892-8287-749bd84a04c2 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.308050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 702/1402] 10.222.0.36 () {54 vars in 912 bytes} [Tue Oct 25 13:24:17 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:17.310342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a5b2b-17c7-4892-8287-749bd84a04c2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.310964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731b38c56a474a70ac8bcfdf641e98a6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:17.325348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731b38c56a474a70ac8bcfdf641e98a6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:17.326499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731b38c56a474a70ac8bcfdf641e98a6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:17.328870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cbb0835f-f0d9-4a70-89ce-53d74b9ec841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731b38c56a474a70ac8bcfdf641e98a6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:17.341331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a5b2b-17c7-4892-8287-749bd84a04c2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.346060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:17.353041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 701/1403] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:17 2022] DELETE /identity/v3/users/731b38c56a474a70ac8bcfdf641e98a6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:17.356550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.357054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.358431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.359538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.359671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.359912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] PATH_INFO: `/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6/users/48fac9275ffa4a0a8b73800dce4b54a6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.360015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.365207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd10e35a6cf4163a7d83cf802c485c6, user_id=48fac9275ffa4a0a8b73800dce4b54a6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.371000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b441d-4c62-4623-b93c-82ea8d7b7256 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.394061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 703/1404] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:17 2022] PUT /identity/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6/users/48fac9275ffa4a0a8b73800dce4b54a6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:17.407375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.408752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.409599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.445644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:17.447076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.480013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] MFA Rules not processed for user `48fac9275ffa4a0a8b73800dce4b54a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:17.488969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.559999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 702/1405] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:17.593806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:17.607451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.610594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.639924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823be36-4fd9-497c-b7ec-bad6cca6973b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.641009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 704/1406] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:17.645031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.645600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.645876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.647226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.663266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.665144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985b16-980d-4c3a-8a7d-56df8b84fece service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.666218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.667854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.691656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 703/1407] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:17.752676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47084ba6-ab4e-4cc5-9771-b3fd66c9b56f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.757246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.757879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.758166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.758435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.768736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:17.770238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23282fbe-ba51-4596-a020-2190bb0a98b8 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:17.794275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 705/1408] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:17.840104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ce5-6e9e-426b-a1a9-4a63af7e5013 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.840479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ce5-6e9e-426b-a1a9-4a63af7e5013 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.840947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ce5-6e9e-426b-a1a9-4a63af7e5013 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.842681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 704/1409] 10.222.0.36 () {54 vars in 912 bytes} [Tue Oct 25 13:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:17.848787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.849170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.849496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.888229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:17.894820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:17.914247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 706/1410] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:17.937720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643ce5-6e9e-426b-a1a9-4a63af7e5013 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:17.956065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643ce5-6e9e-426b-a1a9-4a63af7e5013 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:17.964937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:17.965915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:17.966149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:17.966431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:17.976007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:18.078406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:18.080095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:18.136065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:18.144341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:18.170808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 705/1411] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:18.185394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:18.189820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:18.230778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d1202-b508-4082-8cf1-41187bc81a0e service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:18.235756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:18.236401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:18.236684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:18.236963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:18.253189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:18.254896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:18.273317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bedcc-9f83-46e5-a8dd-3ecfd23afe90 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:18.276891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 706/1412] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:18.284721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:18.285299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:18.285706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:18.286093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] PATH_INFO: `/v3/projects/0be51f2486df4f52861e160ebbe70928` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:18.287998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be51f2486df4f52861e160ebbe70928)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:18.289725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: WARNING py.warnings [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 25 13:24:18.289725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: warnings.warn(msg) Oct 25 13:24:18.289725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: Oct 25 13:24:18.293535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:18.399858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be51f2486df4f52861e160ebbe70928', 'request_id': 'req-a4cc88e8-3fee-4d47-a24d-75d496987c10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.454464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be51f2486df4f52861e160ebbe70928', 'request_id': 'req-a4cc88e8-3fee-4d47-a24d-75d496987c10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.486843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be51f2486df4f52861e160ebbe70928', 'request_id': 'req-a4cc88e8-3fee-4d47-a24d-75d496987c10'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.512729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] The token cache is being invalidate because project 0be51f2486df4f52861e160ebbe70928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:18.514295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.519588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.526446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 707/1413] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:18 2022] DELETE /identity/v3/projects/0be51f2486df4f52861e160ebbe70928 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:18.527781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:18.547706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33fb3-9a30-49f8-b5d4-f42da67f53ad service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:18.598113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:18.600254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:18.693112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:18.747339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:18.748000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:18.748354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:18.748650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] PATH_INFO: `/v3/users/1d310e5e3fb14a0ebf01d822779d1c7e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:18.749829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d310e5e3fb14a0ebf01d822779d1c7e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:18.752063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:18.758921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:18.849209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc88e8-3fee-4d47-a24d-75d496987c10 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:18.860897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:18.861748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:18.862122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:18.862487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] PATH_INFO: `/v3/projects/807408a17e8e4e8786b2f906a126c3d1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:18.873900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807408a17e8e4e8786b2f906a126c3d1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:18.873900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:18.943861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d310e5e3fb14a0ebf01d822779d1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:18.988288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d310e5e3fb14a0ebf01d822779d1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.001184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807408a17e8e4e8786b2f906a126c3d1', 'request_id': 'req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.019588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d310e5e3fb14a0ebf01d822779d1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.020709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d310e5e3fb14a0ebf01d822779d1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.021457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e3b2ef22-bba5-4873-ad03-6d36a595620c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d310e5e3fb14a0ebf01d822779d1c7e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.045923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807408a17e8e4e8786b2f906a126c3d1', 'request_id': 'req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.050947 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 707/1414] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:18 2022] DELETE /identity/v3/users/1d310e5e3fb14a0ebf01d822779d1c7e => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:19.051506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.051721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.051881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.067419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807408a17e8e4e8786b2f906a126c3d1', 'request_id': 'req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.106461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] The token cache is being invalidate because project 807408a17e8e4e8786b2f906a126c3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:19.108066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.110339 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8ba03828-e4ee-4e75-86dc-2b89d6da6f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.118499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 708/1415] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:18 2022] DELETE /identity/v3/projects/807408a17e8e4e8786b2f906a126c3d1 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:19.131412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6c13-b102-47d2-a0c2-4d32e45e4be7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.133001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6c13-b102-47d2-a0c2-4d32e45e4be7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.134721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb6c13-b102-47d2-a0c2-4d32e45e4be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.194666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] MFA Rules not processed for user `28e088c7bf2f41a397d41e8d3c2d7bae`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:19.199805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.212121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-20bb6c13-b102-47d2-a0c2-4d32e45e4be7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:19.216858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20bb6c13-b102-47d2-a0c2-4d32e45e4be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.368149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 708/1416] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:19.369228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:19.372723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.384467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 709/1417] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:19.389910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.391493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.392771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.427202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797fcfd-3c79-4c0d-b41c-8cd2cb5e05eb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:19.429729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:19.431224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.439521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:19.440150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.441315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.441600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] PATH_INFO: `/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.442829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc177b2c142e4ee1a47db065e46b3a5f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:19.445381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:19.463198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 710/1418] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:19.464274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:19.465705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.571987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc177b2c142e4ee1a47db065e46b3a5f', 'request_id': 'req-5e158a5c-dde9-4eaa-b373-62d471d4458c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.587776 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc177b2c142e4ee1a47db065e46b3a5f', 'request_id': 'req-5e158a5c-dde9-4eaa-b373-62d471d4458c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.593775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:19.602327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc177b2c142e4ee1a47db065e46b3a5f', 'request_id': 'req-5e158a5c-dde9-4eaa-b373-62d471d4458c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.620243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] The token cache is being invalidate because project bc177b2c142e4ee1a47db065e46b3a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:19.620941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.622298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:19.623057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:19.630284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 709/1419] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:19 2022] DELETE /identity/v3/projects/bc177b2c142e4ee1a47db065e46b3a5f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:19.631564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:19.635198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.656143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1ab8-dd7f-495c-9b28-35fd50b286e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:19.785677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:19.792195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.792676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.793080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.801503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.858929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158a5c-dde9-4eaa-b373-62d471d4458c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:19.864316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:19.864950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.865228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.865501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.868008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.880986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:19.885004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:19.896773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 710/1420] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:19.898000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:19.899694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:19.918456 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:19.920293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719be9-a518-4417-bf5e-1cf27f6a49b8 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:19.939100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 711/1421] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:19.940644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:19.940929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:19.941201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:19.978304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] MFA Rules not processed for user `213a6f956e5c48b4b45a6481e29db89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:19.981578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:20.010261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e3fb4-305f-4726-9bca-aa70f74b8ac7 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.016507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.029129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.029447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.029722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.030449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:20.030775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.038153 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34431b04-7669-4cd7-8885-6ae1051e3f52 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.118326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 711/1422] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:20.129038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ff05-3a9b-4a0b-95c1-f726b59204db None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.129522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ff05-3a9b-4a0b-95c1-f726b59204db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.132332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12ff05-3a9b-4a0b-95c1-f726b59204db None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.140617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 712/1423] 10.222.0.36 () {54 vars in 961 bytes} [Tue Oct 25 13:24:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:20.144611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d0f0-d998-4b23-9cce-c8ab739837a1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.145008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d0f0-d998-4b23-9cce-c8ab739837a1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.146496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386d0f0-d998-4b23-9cce-c8ab739837a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.178927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 712/1424] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:20.180146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.225666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1070c2e-cdeb-4de6-82bb-b58e545eff69 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.230487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8386d0f0-d998-4b23-9cce-c8ab739837a1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:20.232950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8386d0f0-d998-4b23-9cce-c8ab739837a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:20.259203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 713/1425] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:20.259203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195514-8317-4117-8a9a-7bd961a88410 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.259203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195514-8317-4117-8a9a-7bd961a88410 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.259203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195514-8317-4117-8a9a-7bd961a88410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.267389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.268123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.269012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.271826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.272571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.275288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.306215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-21195514-8317-4117-8a9a-7bd961a88410 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:20.308491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-21195514-8317-4117-8a9a-7bd961a88410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:20.345379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 714/1426] 10.222.0.36 () {58 vars in 1058 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:20.347853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.348147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.348417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.362134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 713/1427] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:20.362134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.380068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:20.402296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:20.403798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:20.560934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 715/1428] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:20.562228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.562642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4831a5-ed66-4bea-b1da-82e0679ebc52 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.576667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ae36e-7f7d-400f-afeb-0304ddae1d9e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.579353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.580105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.580391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.580663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.603544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.604526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.605325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.609420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.609891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.610303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.611103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.614880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.642813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:20.646623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 714/1429] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:20.650957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.680848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 716/1430] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:20.684275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.695424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972997a6-6456-4042-a411-da3ba342d93b service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.709270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03dd01-4f39-4fe8-8f64-315dd377f8a8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.716590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.717228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.717569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.717852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.721446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:20.721783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.722235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.723663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.724950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.726031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.727437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.727838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.731840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.775623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 717/1431] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:20.777832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.809218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 715/1432] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:20.810431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.825529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb5c59-449b-401f-9077-d17c0802f25f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.837069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.838113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.838606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.839032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.840364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.844544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.865014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e2c3e-8ea6-46c9-82b6-08b2e8ba1cb6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:20.905650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:20.908557 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:20.909002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:20.909429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:20.923947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:20.927777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:20.941871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 718/1433] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:20.951036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:20.966209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 716/1434] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:20.969014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:21.028909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865e29-75fb-4790-9222-48260d590820 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:21.156745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a7a8c-fe1d-4d09-82ba-a9e90f0eaf85 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:21.171834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:21.177338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.179137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.179544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.180513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:21.190814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:21.242130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:21.242841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.243120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.248522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] PATH_INFO: `/v3/projects/c3506a5d5a8f47b0a8d570c2af2336a7/users/c6fb41b5706048ef845ff4033ccb171c/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.296049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 719/1435] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:20 2022] POST /identity/v3/users => generated 286 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:21.299375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:21.317990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3506a5d5a8f47b0a8d570c2af2336a7, user_id=c6fb41b5706048ef845ff4033ccb171c, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:21.320379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883173a1-c232-4cd1-bcf6-939e81c2f5e7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:21.335049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125500-4f47-4994-b48a-e9ea26f3e2d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:21.335989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:21.336726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.337101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.337843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.339737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:21.343775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:21.390930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 720/1436] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:21.392333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:21.404560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 717/1437] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:20 2022] PUT /identity/v3/projects/c3506a5d5a8f47b0a8d570c2af2336a7/users/c6fb41b5706048ef845ff4033ccb171c/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:21.416126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.417731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.418014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.422989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac1bb-2916-4dd2-9075-9bcb580ed087 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:21.621950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] MFA Rules not processed for user `c6fb41b5706048ef845ff4033ccb171c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:21.634349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:21.654761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:21.657843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.661436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.661788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.667582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:21.676989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:21.700740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 721/1438] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:21.715454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:21.751379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1193a8-daca-4031-9f7f-93d40bc1a3e3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:21.767983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:21.768682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.769013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.769282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] PATH_INFO: `/v3/projects/d630425d8b514829bd2ffd1fdeb81a54/users/3733063da4fd4f239d079bbbdfb95492/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.797496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d630425d8b514829bd2ffd1fdeb81a54, user_id=3733063da4fd4f239d079bbbdfb95492, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:21.800950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8471bd-4b62-4644-8d13-c43649da2db5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:21.843335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 722/1439] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:21 2022] PUT /identity/v3/projects/d630425d8b514829bd2ffd1fdeb81a54/users/3733063da4fd4f239d079bbbdfb95492/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:21.868053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:21.869500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:21.870114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:21.891357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:21.939437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:21.972911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] MFA Rules not processed for user `3733063da4fd4f239d079bbbdfb95492`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:21.974163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:22.016610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 718/1440] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 603 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:22.021267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:22.089175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70652f-1f77-4cc7-96bf-3619226c90dc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:22.138787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 723/1441] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:22.141235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:22.365420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963788b0-997e-4c76-84f8-ab9cf595d089 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:22.381142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:22.382581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:22.382911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:22.385144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:22.389368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:22.407781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:22.485068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:22.485727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:22.487354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:22.487636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:22.489903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:22.494656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:22.510811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 724/1442] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:22 2022] POST /identity/v3/projects => generated 373 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:22.520224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:22.562496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 719/1443] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:22 2022] POST /identity/v3/projects => generated 363 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:22.568499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:22.780070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172140-d971-49da-a70f-a297eed899b5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:22.903298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50449-ae02-48f4-9bc4-b13f4cfa63cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:22.940004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:22.940665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:22.940980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:22.941233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:22.941891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:22.959993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:22.970429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:23.051628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 720/1444] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:22 2022] POST /identity/v3/users => generated 288 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:23.061802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.062509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.062792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.063061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.063822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.065874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.066217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.110464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42ee1e-8f2c-4bb1-a8c4-d5eb30a0edc1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.116036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 725/1445] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:22 2022] POST /identity/v3/users => generated 293 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:23.119532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.154109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.154790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.155079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.156315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.162514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.162926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf771b12-d6b6-4c4c-b6c2-6bcf5f7f6d36 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.167618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.188848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.189499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.189780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.190048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.193987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.199933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.200052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 721/1446] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:23.205451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.230429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 726/1447] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:23.234692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.248208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eec467-aa16-4b39-a258-fa62ecaf04aa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.261528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.262224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.262513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.262824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] PATH_INFO: `/v3/projects/3464976e3e024c17ad15685e8af6a061/users/64f392b2e516472db6cfd941a87fd70b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.286051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fa7bd-2e3e-403e-bb3a-05ccb288e429 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.301675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.302386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.302793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.303137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] PATH_INFO: `/v3/projects/b33b21cefab34262873f7ad3d65982cb/users/67ea7cdc1c4c4c92b5897df31db714fa/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.335004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3464976e3e024c17ad15685e8af6a061, user_id=64f392b2e516472db6cfd941a87fd70b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.346659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.352569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33b21cefab34262873f7ad3d65982cb, user_id=67ea7cdc1c4c4c92b5897df31db714fa, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.355749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.389561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 727/1448] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:23 2022] PUT /identity/v3/projects/b33b21cefab34262873f7ad3d65982cb/users/67ea7cdc1c4c4c92b5897df31db714fa/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:23.393884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.410197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 722/1449] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:23 2022] PUT /identity/v3/projects/3464976e3e024c17ad15685e8af6a061/users/64f392b2e516472db6cfd941a87fd70b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:23.411232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.465693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:23.554273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d01b2-0c22-4df0-9703-0543128d270e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.563593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.564247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.564517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.564787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.565445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.575774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.594930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 728/1450] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:23.607006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.609893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e733f80-2701-4e19-af0e-33013b26f251 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.631474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.632260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.632643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.633016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.633782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.637260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.658295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6990146-0108-4458-96d9-4b82dd4fe772 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.672757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 723/1451] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:23.676891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:23.678701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.678984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.679252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] PATH_INFO: `/v3/projects/b33b21cefab34262873f7ad3d65982cb/users/67ea7cdc1c4c4c92b5897df31db714fa/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.688548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33b21cefab34262873f7ad3d65982cb, user_id=67ea7cdc1c4c4c92b5897df31db714fa, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:23.692670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2dc4-aeeb-4ed0-ace6-5ae264843f3b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:23.698006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:23.723136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc7a9-0c12-4285-b75c-c106097dc8dd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:23.732145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 729/1452] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:23 2022] PUT /identity/v3/projects/b33b21cefab34262873f7ad3d65982cb/users/67ea7cdc1c4c4c92b5897df31db714fa/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:23.740267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:23.741855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:23.741855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:23.836287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] MFA Rules not processed for user `67ea7cdc1c4c4c92b5897df31db714fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:23.843490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:23.989355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:24.000703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:24.001997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:24.003905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:24.004382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] PATH_INFO: `/v3/projects/3464976e3e024c17ad15685e8af6a061/users/64f392b2e516472db6cfd941a87fd70b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:24.007078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3464976e3e024c17ad15685e8af6a061, user_id=64f392b2e516472db6cfd941a87fd70b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:24.008321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 730/1453] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:24.010514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ab092-f4b4-4163-92f0-f33ec9290ba8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:24.035497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:24.036948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:24.041860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 724/1454] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:23 2022] PUT /identity/v3/projects/3464976e3e024c17ad15685e8af6a061/users/64f392b2e516472db6cfd941a87fd70b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:24.052311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:24.059261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:24.059879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:24.132045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] MFA Rules not processed for user `64f392b2e516472db6cfd941a87fd70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:24.134049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:24.170166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d245-e394-48ff-92fb-2bc4bef043b5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:24.282932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:24.283936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:24.284273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:24.284604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:24.323052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 725/1455] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:24.441347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:24.451206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:24.475482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 731/1456] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:24.561801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:24.577919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e55cf-8cc3-4a09-95bc-cbb5a10316e2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:24.597677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:24.598327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:24.598602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:24.598874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:24.625861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:24.627401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf85df9-4280-46e1-b354-4cea61203c7e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:24.649977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 726/1457] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:24.852959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:24.854725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:24.870843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d567-187a-4cd6-9b2b-55a3655c0758 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:24.876751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:24.877416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:24.877685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:24.877955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:25.004038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a5cf-385e-469f-821b-33cad90a3319 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:25.004590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a5cf-385e-469f-821b-33cad90a3319 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:25.004590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2a5cf-385e-469f-821b-33cad90a3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:25.029694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:25.034018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:25.040574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:25.054862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e1a2a5cf-385e-469f-821b-33cad90a3319 None None] MFA Rules not processed for user `28e088c7bf2f41a397d41e8d3c2d7bae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:25.056673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1a2a5cf-385e-469f-821b-33cad90a3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:25.065917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 732/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:25.083623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:25.188541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 727/1459] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:25.708015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:25.792973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a0b7a-aff4-4cdb-bb1d-73f66e4bb82d service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:25.798819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:25.799174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:25.799822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:25.800163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] PATH_INFO: `/v3/users/3733063da4fd4f239d079bbbdfb95492` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:25.801603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3733063da4fd4f239d079bbbdfb95492)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:25.803655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:25.982863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3733063da4fd4f239d079bbbdfb95492'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:26.003044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3733063da4fd4f239d079bbbdfb95492'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:26.016858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3733063da4fd4f239d079bbbdfb95492'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:26.017700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3733063da4fd4f239d079bbbdfb95492'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:26.018449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3733063da4fd4f239d079bbbdfb95492'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:26.037629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 733/1460] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:25 2022] DELETE /identity/v3/users/3733063da4fd4f239d079bbbdfb95492 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:26.042098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:26.043255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:26.043605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:26.065046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:26.066299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:26.162570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 728/1461] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:26.178954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:26.181160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:26.259457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b233a-7683-4dd7-bfd2-9f0978838565 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:26.263360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:26.263985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:26.264264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:26.264531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:26.275309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:26.277045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7de07-f0fd-48ea-a47f-e4c3b472fab3 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:26.294200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 734/1462] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:27.209045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:27.212064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:27.237788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2b70c-9812-4674-a3c6-a2c72ff432ab None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:27.280087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:27.282535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:27.285531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:27.285970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] PATH_INFO: `/v3/users/0014081382b241e7b6ba9ea349b41c7d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:27.296055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0014081382b241e7b6ba9ea349b41c7d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:27.298333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:27.360806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:27.361328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:27.361441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:27.423772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:27.465692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0014081382b241e7b6ba9ea349b41c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:27.480721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0014081382b241e7b6ba9ea349b41c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:27.481652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0014081382b241e7b6ba9ea349b41c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:27.494751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0014081382b241e7b6ba9ea349b41c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:27.497119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7b3ea335-d75f-4c7f-891a-5e4889ad9e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0014081382b241e7b6ba9ea349b41c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:27.502487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:27.504718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:27.541907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 729/1463] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:27 2022] DELETE /identity/v3/users/0014081382b241e7b6ba9ea349b41c7d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:27.544968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:27.546111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:27.547894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:27.621979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:27.640200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:27.782004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 735/1464] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:24:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:27.782146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:27.867709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 730/1465] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:27.953595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:27.955115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:27.977500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34748ed0-fdc6-4181-ae6a-98060f0e7086 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:28.085938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:28.087655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:28.173144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:28.173854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:28.175748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:28.176029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:28.210062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:28.211587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:28.245041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 736/1466] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:28.254531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:28.257242 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:28.307592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e81f5-6a53-45c5-be8e-bdea3aa9b0ad service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:28.323849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:28.327948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:28.329099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:28.329974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] PATH_INFO: `/v3/projects/d630425d8b514829bd2ffd1fdeb81a54` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:28.335865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d630425d8b514829bd2ffd1fdeb81a54)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:28.350999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:28.442746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eac73-1da0-4eb2-a9e3-4f331946eaf1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:28.458797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:28.465955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:28.468483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:28.469077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:28.514382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd630425d8b514829bd2ffd1fdeb81a54', 'request_id': 'req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:28.528638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd630425d8b514829bd2ffd1fdeb81a54', 'request_id': 'req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:28.547257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd630425d8b514829bd2ffd1fdeb81a54', 'request_id': 'req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:28.595498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] The token cache is being invalidate because project d630425d8b514829bd2ffd1fdeb81a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:28.597098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:28.599447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:28.606858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 737/1467] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:28 2022] DELETE /identity/v3/projects/d630425d8b514829bd2ffd1fdeb81a54 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:28.610996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:28.631455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:28.656183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:28.658234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:28.671882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:28.674585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 731/1468] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:24:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 25 13:24:28.810717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84252d-600a-40f2-9d8f-c6e409cc4dfc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:28.836248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:28.839171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:28.839519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:28.839922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] PATH_INFO: `/v3/users/67ea7cdc1c4c4c92b5897df31db714fa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:28.844051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ea7cdc1c4c4c92b5897df31db714fa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:28.851434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:28.997950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ea7cdc1c4c4c92b5897df31db714fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:29.033458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ea7cdc1c4c4c92b5897df31db714fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:29.051747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ea7cdc1c4c4c92b5897df31db714fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:29.052875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ea7cdc1c4c4c92b5897df31db714fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:29.054543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ea7cdc1c4c4c92b5897df31db714fa'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:29.087478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 738/1469] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:28 2022] DELETE /identity/v3/users/67ea7cdc1c4c4c92b5897df31db714fa => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:29.594133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:29.596055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:29.645286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa42a4-0c74-4b30-9399-0d3eabc0dc0e service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:29.725757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:29.727884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:29.797466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:29.800130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:29.800702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:29.801114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] PATH_INFO: `/v3/users/054d87c1c9f4444184c8b00b5e3d0dd9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:29.803212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054d87c1c9f4444184c8b00b5e3d0dd9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:29.809610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:29.834606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:29.982683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054d87c1c9f4444184c8b00b5e3d0dd9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.044455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054d87c1c9f4444184c8b00b5e3d0dd9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.049404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054d87c1c9f4444184c8b00b5e3d0dd9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.079064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054d87c1c9f4444184c8b00b5e3d0dd9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.083963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054d87c1c9f4444184c8b00b5e3d0dd9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.149735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325f07f-435a-40df-856d-0eb5566c8b12 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:30.157542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 732/1470] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:29 2022] DELETE /identity/v3/users/054d87c1c9f4444184c8b00b5e3d0dd9 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:30.158480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:30.172312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:30.341092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:30.375136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:30.375136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.375136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.375597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] PATH_INFO: `/v3/users/c6fb41b5706048ef845ff4033ccb171c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.377003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fb41b5706048ef845ff4033ccb171c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:30.379042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:30.494737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37f7ab-08d7-49e3-bf33-adbaa9249336 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:30.514605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:30.515397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.515677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.515952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] PATH_INFO: `/v3/projects/711e6c16de3f4930857f28814947f041` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.517875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711e6c16de3f4930857f28814947f041)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:30.519922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:30.627999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fb41b5706048ef845ff4033ccb171c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.668236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711e6c16de3f4930857f28814947f041', 'request_id': 'req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.681458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fb41b5706048ef845ff4033ccb171c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.685078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711e6c16de3f4930857f28814947f041', 'request_id': 'req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.698256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fb41b5706048ef845ff4033ccb171c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.700234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fb41b5706048ef845ff4033ccb171c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.700358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7291b4e6-c263-4bcb-afad-654ccb170c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fb41b5706048ef845ff4033ccb171c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.714156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711e6c16de3f4930857f28814947f041', 'request_id': 'req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.734639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 739/1471] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:29 2022] DELETE /identity/v3/users/c6fb41b5706048ef845ff4033ccb171c => generated 0 bytes in 1009 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:30.736598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c1c6a-069b-4226-9220-64f312b1d478 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.737001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c1c6a-069b-4226-9220-64f312b1d478 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.737555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c1c6a-069b-4226-9220-64f312b1d478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.753930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] The token cache is being invalidate because project 711e6c16de3f4930857f28814947f041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:30.754224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.757576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2a3d8cfd-b302-4a2c-a42c-63d5d3cd2f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:30.759883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 733/1472] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:30 2022] DELETE /identity/v3/projects/711e6c16de3f4930857f28814947f041 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:30.761172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.764216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.765330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.771974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a75c1c6a-069b-4226-9220-64f312b1d478 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:30.774767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a75c1c6a-069b-4226-9220-64f312b1d478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:30.807030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:30.808752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:30.815410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:30.918194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 740/1473] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:30.921206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.921735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.923895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.933813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 734/1474] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 25 13:24:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:30.934958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:30.941126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:30.982154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94197aea-cac5-4a47-adfe-9efcd4e94fb8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:30.994668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:30.995627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:30.996045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:30.996497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] PATH_INFO: `/v3/users/48fac9275ffa4a0a8b73800dce4b54a6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:30.998517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48fac9275ffa4a0a8b73800dce4b54a6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.006112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.141457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48fac9275ffa4a0a8b73800dce4b54a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:31.188043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48fac9275ffa4a0a8b73800dce4b54a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:31.188043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48fac9275ffa4a0a8b73800dce4b54a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:31.213267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48fac9275ffa4a0a8b73800dce4b54a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:31.216772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48fac9275ffa4a0a8b73800dce4b54a6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:31.254225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 735/1475] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:30 2022] DELETE /identity/v3/users/48fac9275ffa4a0a8b73800dce4b54a6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:31.256175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.264142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.282798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:31.284407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.393515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 741/1476] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:31.395601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.401771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.417987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:31.485632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c06d22-f5fb-451c-ba4d-216910cc2d6f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.514684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.517834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.518277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.520024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.539853 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.596625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.608575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.656264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 736/1477] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:31.657184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.700370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7fd6f-ceaa-4de1-9e1b-c536e7e18295 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.715386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2dfe2-5d31-464c-bfcc-0ec09c1b3aac service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.717854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.718841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.719162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.721447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.728862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.729773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.734323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.736110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.737207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.755314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.757194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.793041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.795208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 742/1478] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:31.796303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.800686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.813399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142fbd2-2525-417e-bb4d-04bf418d03bf service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.823564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.824678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.825475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.825907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.844178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 737/1479] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:31.846950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.853424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.854747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.856890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.878456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 743/1480] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:31.883229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:31.887609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:31.906204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3baf12-9158-4021-9c94-2bb4217d07fc service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.914564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cf457-0a94-438f-a16e-ccb550d22eeb service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:31.926095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.928629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:31.928796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.928993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.929278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] PATH_INFO: `/v3/projects/b33b21cefab34262873f7ad3d65982cb` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.931003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:31.931641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:31.931756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33b21cefab34262873f7ad3d65982cb)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.933532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] PATH_INFO: `/v3/users/da49ebb09988435d96f27fdaca42918b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:31.935407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.936074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da49ebb09988435d96f27fdaca42918b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:31.943420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:31.994253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:32.063428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da49ebb09988435d96f27fdaca42918b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.083567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33b21cefab34262873f7ad3d65982cb', 'request_id': 'req-22fc3031-527c-4234-9af5-072d43cb8b28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.098030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da49ebb09988435d96f27fdaca42918b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.111178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33b21cefab34262873f7ad3d65982cb', 'request_id': 'req-22fc3031-527c-4234-9af5-072d43cb8b28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.118226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da49ebb09988435d96f27fdaca42918b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.119434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da49ebb09988435d96f27fdaca42918b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.120404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-1511d864-9f80-4011-9db8-e907badc0ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da49ebb09988435d96f27fdaca42918b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.126417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33b21cefab34262873f7ad3d65982cb', 'request_id': 'req-22fc3031-527c-4234-9af5-072d43cb8b28'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.151567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 744/1481] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:31 2022] DELETE /identity/v3/users/da49ebb09988435d96f27fdaca42918b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:32.159753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] The token cache is being invalidate because project b33b21cefab34262873f7ad3d65982cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:32.160058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.160681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4e41c-04bd-4cf1-a34d-30aece88379f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.161173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4e41c-04bd-4cf1-a34d-30aece88379f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.161634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4e41c-04bd-4cf1-a34d-30aece88379f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.162562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-22fc3031-527c-4234-9af5-072d43cb8b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:32.164556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 738/1482] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:31 2022] DELETE /identity/v3/projects/b33b21cefab34262873f7ad3d65982cb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:32.174581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.174890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.175176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.223214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-46e4e41c-04bd-4cf1-a34d-30aece88379f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:32.224002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-46e4e41c-04bd-4cf1-a34d-30aece88379f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:32.234819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:32.236058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:32.347037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 745/1483] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:32.417289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 739/1484] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:32.424676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.425012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.425287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.430798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:32.432758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:32.468762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:32.487575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:32.488864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:32.505677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 746/1485] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:32.511745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:32.532572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d4f56-d914-4bc8-aedb-4503da6e89b8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:32.545097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:32.545762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.546040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.546347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.547466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:32.547808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:32.549819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:32.574681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 747/1486] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:32.586033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:32.601687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa39af-3b01-4feb-8b11-7ba87bd8c018 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:32.609371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574856-9d11-4f74-97e0-64f63f2dbbb4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:32.611428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:32.612132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.612478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.612766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.633113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:32.633873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.637326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.637655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.639685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:32.645062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:32.656983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:32.658561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:32.678152 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 740/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:32.713779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 748/1488] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:32 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:32.718561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:32.731548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a30a0-d218-4346-889b-78064ead380e service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:32.896313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:32.932308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:32.934352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:32.935126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:32.936055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:32.937674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:32.941921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:32.983733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 741/1489] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:32 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:32.988717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.009965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d9064-296e-42e6-b528-74f92be5d62f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:33.027522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:33.028436 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.028750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.029196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.029963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:33.031970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:33.061977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.062433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 749/1490] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:33.080475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce88c4-10f2-42a0-9335-26dfa367149a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:33.089321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:33.090420 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.090835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.092145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.093635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:33.096826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990fda-f7f7-484f-8203-63e306370efc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:33.115587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 742/1491] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:33.121266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.139249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d1fd7-7b83-46f4-9a50-95920d27da32 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:33.154394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:33.157420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.158838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.159700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] PATH_INFO: `/v3/projects/02f70c4d586949678f615292c7d49957/users/8b3d2738dc60481c963bc18ce3dcd2bf/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.183636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f70c4d586949678f615292c7d49957, user_id=8b3d2738dc60481c963bc18ce3dcd2bf, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:33.189542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:33.240064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 750/1492] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:33 2022] PUT /identity/v3/projects/02f70c4d586949678f615292c7d49957/users/8b3d2738dc60481c963bc18ce3dcd2bf/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:33.242128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.242610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.244968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.313989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] MFA Rules not processed for user `8b3d2738dc60481c963bc18ce3dcd2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:33.322693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:33.522393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 743/1493] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:33.551192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.772918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1cee-4908-43b0-96e1-850fe8998676 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:33.789379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:33.790318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.790802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.791183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.792157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:33.796687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:33.850763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 751/1494] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:33 2022] POST /identity/v3/projects => generated 387 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:33.852065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.868439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a0bcf-2a89-4c81-b8af-ba94dd3319af None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:33.902856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:33.904781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:33.985292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:33.986621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:33.987061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:33.987451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:33.988221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:33.991730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.044507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 744/1495] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:33 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:34.049708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.071731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fef4bb-5192-4736-81c8-3de1bc3e5d24 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.080677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.082509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.085001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.086046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.086827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.089541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.103276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7f4b9-a122-4081-955d-fb21ebfc20ae admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.113305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 745/1496] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:34.117320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.117508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.118050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.118332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.118607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] PATH_INFO: `/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.122683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd10e35a6cf4163a7d83cf802c485c6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.124734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.143898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a5b96-975d-40d6-8f17-d0dcb9bc6eb3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.151600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.152294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.152598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.152871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.153530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.155521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.168477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:34.180604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 746/1497] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:34.184352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.203931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe0af5-1547-4b9a-8db5-a3a81244a5fd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.212034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.212932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.213116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.213391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] PATH_INFO: `/v3/projects/ee02ecb430ba403da81000322dedfdf6/users/7a4afb5d615c4679a41f04ae67e974de/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.245385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee02ecb430ba403da81000322dedfdf6, user_id=7a4afb5d615c4679a41f04ae67e974de, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.246126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd10e35a6cf4163a7d83cf802c485c6', 'request_id': 'req-02daafcb-3022-409a-8ea6-8e8aec2f0f03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.249607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e0bf9-c2e7-4fb7-8aed-3d4164cd5677 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.262469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd10e35a6cf4163a7d83cf802c485c6', 'request_id': 'req-02daafcb-3022-409a-8ea6-8e8aec2f0f03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.264774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:34.283199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 747/1498] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:34 2022] PUT /identity/v3/projects/ee02ecb430ba403da81000322dedfdf6/users/7a4afb5d615c4679a41f04ae67e974de/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:34.287113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd10e35a6cf4163a7d83cf802c485c6', 'request_id': 'req-02daafcb-3022-409a-8ea6-8e8aec2f0f03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.291372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.291675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.291979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.305277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] The token cache is being invalidate because project fdd10e35a6cf4163a7d83cf802c485c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:34.306085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.307923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-02daafcb-3022-409a-8ea6-8e8aec2f0f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.310904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 752/1499] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:33 2022] DELETE /identity/v3/projects/fdd10e35a6cf4163a7d83cf802c485c6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:34.317261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ea17-5be2-4987-ad24-91845ea60a06 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.317944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ea17-5be2-4987-ad24-91845ea60a06 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.318390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ea17-5be2-4987-ad24-91845ea60a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.356074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7111ea17-5be2-4987-ad24-91845ea60a06 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:34.357796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7111ea17-5be2-4987-ad24-91845ea60a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.378336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] MFA Rules not processed for user `7a4afb5d615c4679a41f04ae67e974de`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:34.383915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.477124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 753/1500] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:34.499480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.500177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.501942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.533997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:34.535526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.570978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 754/1501] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:34.575402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.584017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.620582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6fdbb-4bdb-4078-aa0f-385a380fba32 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.621344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 748/1502] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:34.622342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.636657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a327ae-ad1d-460c-8e92-3039e23116f4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.643758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.644604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.644979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.645403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.646369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:34.646689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.647450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.648454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.648454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.648665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] PATH_INFO: `/v3/projects/b1637e3f23174ca1803584e65bef1843` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.648747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.659256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1637e3f23174ca1803584e65bef1843)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.663771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 749/1503] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:34.664736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.666971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.671407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.686591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdee790-c215-43ee-be2a-90df5d0cce6d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.697044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:34.698173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:34.698630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:34.699093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:34.700363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:34.703794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:34.762487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 750/1504] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:34.763625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.833938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1637e3f23174ca1803584e65bef1843', 'request_id': 'req-03e02acb-8382-465f-a1a1-4313bb033ba2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.856008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1637e3f23174ca1803584e65bef1843', 'request_id': 'req-03e02acb-8382-465f-a1a1-4313bb033ba2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.878614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928da2f-1c98-48c1-952c-46e7086302ef admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:34.879382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1637e3f23174ca1803584e65bef1843', 'request_id': 'req-03e02acb-8382-465f-a1a1-4313bb033ba2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.917706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] The token cache is being invalidate because project b1637e3f23174ca1803584e65bef1843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:34.918007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.920911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:34.925239 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 755/1505] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:34 2022] DELETE /identity/v3/projects/b1637e3f23174ca1803584e65bef1843 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:34.931354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:34.935000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:34.982222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:35.048758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.048758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.048758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.048758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.049172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.059425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:35.078424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:35.117203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02acb-8382-465f-a1a1-4313bb033ba2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:35.137569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.138859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 751/1506] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/projects => generated 381 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:35.139185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.140214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:35.141373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.141751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.144164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.144764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:35.150004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:35.220196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 756/1507] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:34 2022] POST /identity/v3/users => generated 300 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:35.221173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:35.224134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:35.251744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8b9e-5ca4-435a-ab72-d849ce3b2b3e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:35.350322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c0efd-f8c4-4a8c-abe5-fb55e2d11e9a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:35.364233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.365259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.366390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.367221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] PATH_INFO: `/v3/projects/c905bb2151134069b7eb82242f816236` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.370876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c905bb2151134069b7eb82242f816236)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.375381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:35.439094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.441400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.443499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.443959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] PATH_INFO: `/v3/users/68f6f3ec94c5423c8f64c17870dd5c47` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.446523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f6f3ec94c5423c8f64c17870dd5c47)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.450986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:35.524364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c905bb2151134069b7eb82242f816236', 'request_id': 'req-f6facfe7-64b6-41cb-a7ce-874552c8c051'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.548290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c905bb2151134069b7eb82242f816236', 'request_id': 'req-f6facfe7-64b6-41cb-a7ce-874552c8c051'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.575910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c905bb2151134069b7eb82242f816236', 'request_id': 'req-f6facfe7-64b6-41cb-a7ce-874552c8c051'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.604227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f6f3ec94c5423c8f64c17870dd5c47'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.632898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] The token cache is being invalidate because project c905bb2151134069b7eb82242f816236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:35.633434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.634316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.651612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 752/1508] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:35 2022] DELETE /identity/v3/projects/c905bb2151134069b7eb82242f816236 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:35.655711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:35.656114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f6f3ec94c5423c8f64c17870dd5c47'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.657529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:35.668186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f6f3ec94c5423c8f64c17870dd5c47'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.671676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f6f3ec94c5423c8f64c17870dd5c47'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.672440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f6f3ec94c5423c8f64c17870dd5c47'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:35.724844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 757/1509] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:35 2022] DELETE /identity/v3/users/68f6f3ec94c5423c8f64c17870dd5c47 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:35.724844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:35.731758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:35.774751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facfe7-64b6-41cb-a7ce-874552c8c051 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:35.809739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:35.858405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:35.911325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f36b7-2c82-4e45-a409-d6ce1ab57df5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:35.927114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.928063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.928473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.928903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] PATH_INFO: `/v3/projects/c3506a5d5a8f47b0a8d570c2af2336a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.930548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3506a5d5a8f47b0a8d570c2af2336a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.933904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:35.941423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:35.943227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:35.943679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:35.944285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:35.945252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:35.948023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.001333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 753/1510] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:35 2022] POST /identity/v3/users => generated 291 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:36.003657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.011296 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.045981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668552a-a865-4ef7-8dfa-5576e03f27a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.144721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3506a5d5a8f47b0a8d570c2af2336a7', 'request_id': 'req-a003894d-39c6-41b9-a86d-b38b2bc00bcc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.195880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3506a5d5a8f47b0a8d570c2af2336a7', 'request_id': 'req-a003894d-39c6-41b9-a86d-b38b2bc00bcc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.232323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3506a5d5a8f47b0a8d570c2af2336a7', 'request_id': 'req-a003894d-39c6-41b9-a86d-b38b2bc00bcc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.248200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.248870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.249151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.249421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.250087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.251757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.271269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] The token cache is being invalidate because project c3506a5d5a8f47b0a8d570c2af2336a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:36.273372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 754/1511] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:36.274318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.277476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.277728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.279560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.285679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 758/1512] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:35 2022] DELETE /identity/v3/projects/c3506a5d5a8f47b0a8d570c2af2336a7 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:36.290922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.298916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.299053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8e9b-90d2-42ab-b8bc-aef5959f85fe admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.309456 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.310216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.310541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.310809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] PATH_INFO: `/v3/users/5a8db1c37ff841118b479e52cef4b13e` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.312723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8db1c37ff841118b479e52cef4b13e)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.316023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.347340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003894d-39c6-41b9-a86d-b38b2bc00bcc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.385558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.387408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.389131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8db1c37ff841118b479e52cef4b13e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.389756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.391628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.395960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.402624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.407652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8db1c37ff841118b479e52cef4b13e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.408731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8db1c37ff841118b479e52cef4b13e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.419912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8db1c37ff841118b479e52cef4b13e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.420741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8db1c37ff841118b479e52cef4b13e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.441210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 755/1513] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:36 2022] DELETE /identity/v3/users/5a8db1c37ff841118b479e52cef4b13e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:36.442508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.443889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.467446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 759/1514] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:36.473893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.480854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:36.488725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.537600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8d781-f477-400a-abd3-06fb330213d9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.541675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb9fbc-627e-41b9-b326-7317ed913b55 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.544256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.544945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.545488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.545695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] PATH_INFO: `/v3/projects/05b1030d830e48818710063e024f7e49/users/ff466b8e7b1b4541b23b40a7ed968080/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.564603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b1030d830e48818710063e024f7e49, user_id=ff466b8e7b1b4541b23b40a7ed968080, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.567637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d596099-8d7c-4dd7-878b-e06da7d8422e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.569305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.572180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.573356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.574212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] PATH_INFO: `/v3/users/64f392b2e516472db6cfd941a87fd70b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.580435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f392b2e516472db6cfd941a87fd70b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.595265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.602131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 756/1515] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:36 2022] PUT /identity/v3/projects/05b1030d830e48818710063e024f7e49/users/ff466b8e7b1b4541b23b40a7ed968080/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:36.604143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.604629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.605316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.607237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 757/1516] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:36.608196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.610212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.698785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930b9db-323d-4735-a2c0-e3e30a9bfcc7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.705980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.707329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.707788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.708430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.709647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.711759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.728023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 758/1517] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:36.729426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.741995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d160f-a874-42c7-a52c-485cef4f46f0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.749041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.750111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.750625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.751057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.751948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.754473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455948d-dc8a-4dd4-8271-7c3a53a625d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.776350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 759/1518] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:36.778025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a7ee-0900-45ad-acd2-2f3dff033def None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.778473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a7ee-0900-45ad-acd2-2f3dff033def None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.778911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10a7ee-0900-45ad-acd2-2f3dff033def None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.780636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 760/1519] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:36.782064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10a7ee-0900-45ad-acd2-2f3dff033def None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.783997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f392b2e516472db6cfd941a87fd70b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.799357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f392b2e516472db6cfd941a87fd70b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.803026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10a7ee-0900-45ad-acd2-2f3dff033def None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:36.806637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f392b2e516472db6cfd941a87fd70b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.807458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f392b2e516472db6cfd941a87fd70b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.808857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f392b2e516472db6cfd941a87fd70b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:36.809528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:36.810622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.811088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.811759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] PATH_INFO: `/v3/projects/180ee387dc6541548f5f4d809e54fb3d/users/7aeaf5cb88ee4be89ad906f1e1522840/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.830568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 760/1520] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:36 2022] DELETE /identity/v3/users/64f392b2e516472db6cfd941a87fd70b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:36.830568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:36.830568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.836633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] RBAC: Authorizing `identity:create_grant(project_id=180ee387dc6541548f5f4d809e54fb3d, user_id=7aeaf5cb88ee4be89ad906f1e1522840, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:36.839984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029e3cd-5888-4067-91d9-cc119e61912d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:36.861178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 761/1521] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:36 2022] PUT /identity/v3/projects/180ee387dc6541548f5f4d809e54fb3d/users/7aeaf5cb88ee4be89ad906f1e1522840/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:36.864037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c8df-524d-4e53-a344-32edc65417ab None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:36.864464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:36.865603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c8df-524d-4e53-a344-32edc65417ab None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:36.866081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c8df-524d-4e53-a344-32edc65417ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:36.898300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a494c8df-524d-4e53-a344-32edc65417ab None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:36.899910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a494c8df-524d-4e53-a344-32edc65417ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:36.917642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adacbf08-ecdb-4763-9dec-b88e573cc661 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.008019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:37.024228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 762/1522] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:24:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:37.026517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593f30-b2e9-4c5d-bd45-ec77e1ee8022 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.026993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593f30-b2e9-4c5d-bd45-ec77e1ee8022 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.027601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05593f30-b2e9-4c5d-bd45-ec77e1ee8022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.075957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:37.076662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.076942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.077213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] PATH_INFO: `/v3/projects/05b1030d830e48818710063e024f7e49/users/ff466b8e7b1b4541b23b40a7ed968080/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.082248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b1030d830e48818710063e024f7e49, user_id=ff466b8e7b1b4541b23b40a7ed968080, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:37.085229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab56d1-8276-4c47-8c1a-bcf5403608c2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:37.088251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-05593f30-b2e9-4c5d-bd45-ec77e1ee8022 None None] MFA Rules not processed for user `7aeaf5cb88ee4be89ad906f1e1522840`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:37.096805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-05593f30-b2e9-4c5d-bd45-ec77e1ee8022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.137755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 761/1523] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:36 2022] PUT /identity/v3/projects/05b1030d830e48818710063e024f7e49/users/ff466b8e7b1b4541b23b40a7ed968080/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:37.139544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.142235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.143519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.191173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:37.196299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.262973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 763/1524] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:37.264610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.264982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.265330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.302162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] MFA Rules not processed for user `ff466b8e7b1b4541b23b40a7ed968080`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:37.303622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.373609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 762/1525] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:37.374738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:37.377094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.475888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:37.477887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 764/1526] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:37.493432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:37.495582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bfe09-60ec-4606-aa56-d8ab4580e53a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.497563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.511951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:37.512599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.512885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.513162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.513801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:37.519344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:37.563408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 763/1527] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:37 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:37.567447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:37.623608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56c085-9317-41a0-a1d7-c4be3602acf6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.665774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:37.704216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423b47e-05e1-4502-9259-5c8f7b79c92d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.713703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:37.717479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.718883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.719477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.720061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:37.720588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.720898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.721137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.723579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:37.728084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:37.777236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 764/1528] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:37 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:37.778301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:37.782977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:37.808872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f6ce7-9dfa-402d-abe7-3cfcfd4f6624 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.822602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:37.824948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:37.825144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:37.825478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] PATH_INFO: `/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:37.831015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ff0ee0bbd044acb4ef1c7f8877bb54)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:37.835286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:37.899372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:37.902994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:37.936809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 765/1529] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:37.939002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:37.964021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253956aa-9c24-498c-844c-132d53d4ce99 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:37.968576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ff0ee0bbd044acb4ef1c7f8877bb54', 'request_id': 'req-c03bad98-2c89-48e8-afc7-96486e32467c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:37.986675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ff0ee0bbd044acb4ef1c7f8877bb54', 'request_id': 'req-c03bad98-2c89-48e8-afc7-96486e32467c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:38.003230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ff0ee0bbd044acb4ef1c7f8877bb54', 'request_id': 'req-c03bad98-2c89-48e8-afc7-96486e32467c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:38.034141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] The token cache is being invalidate because project b5ff0ee0bbd044acb4ef1c7f8877bb54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:38.034501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:38.037831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:38.043253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 765/1530] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:37 2022] DELETE /identity/v3/projects/b5ff0ee0bbd044acb4ef1c7f8877bb54 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:38.046608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:38.049493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.116595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:38.117245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.117574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.117847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.118633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:38.120832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b131b-3ccd-41b5-b39b-5e9eec8d5d3f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:38.143819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 766/1531] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:38.145240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.145550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.145822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.178585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:38.180165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.201029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 767/1532] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:38.202534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:38.206220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.247537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12163cf6-3ea5-4726-ac02-0ad5b45b89a4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:38.277888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:38.281055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:38.281937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.282374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.282760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] PATH_INFO: `/v3/projects/30a568656cb840fab636552c55776d94/users/4baf32d185fb4396af16dc42b97712b0/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.344429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bad98-2c89-48e8-afc7-96486e32467c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:38.345060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a568656cb840fab636552c55776d94, user_id=4baf32d185fb4396af16dc42b97712b0, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:38.360372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:38.361662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.362137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.362599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.364761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d8b67-ca50-49c1-8edc-3835fa12f1d6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:38.368181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.410426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 768/1533] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:38 2022] PUT /identity/v3/projects/30a568656cb840fab636552c55776d94/users/4baf32d185fb4396af16dc42b97712b0/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:38.416183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:38.418213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:38.418318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.418583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.418859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.447565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 766/1534] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:38.451617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:38.478783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd90ac5-915a-4322-9352-260be12a9a10 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:38.490423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:38.491947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.686002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:38.686654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.686934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.687234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.687900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:38.691104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:38.698707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:38.739662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 767/1535] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:38.740890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:38.749963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:38.761172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 769/1536] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:38.762183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:38.794412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e64df-ce0f-42b6-8c40-770940809e0d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:38.818163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:38.818904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:38.819215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:38.820138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] PATH_INFO: `/v3/projects/30a568656cb840fab636552c55776d94/users/4baf32d185fb4396af16dc42b97712b0/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:38.832855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a568656cb840fab636552c55776d94, user_id=4baf32d185fb4396af16dc42b97712b0, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:38.839542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:38.876516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 770/1537] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:38 2022] PUT /identity/v3/projects/30a568656cb840fab636552c55776d94/users/4baf32d185fb4396af16dc42b97712b0/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:38.887241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.024304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4edbc-cbdc-4b5b-af18-dd70731b48dd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.082614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864fdb-90bb-42af-911e-9455e324e496 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.109614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.113638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.114307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.114795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.115698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:39.116988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.128844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d1c40-c802-4eb8-abbf-4e845f1434d5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:39.183516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 771/1538] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:39.186100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.186100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.186365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.203962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.204698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.204990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.205369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.211233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.255708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.261182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:39.272125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] MFA Rules not processed for user `4baf32d185fb4396af16dc42b97712b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:39.287235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.308730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:39.335708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 768/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:39.336535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.355970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7376b-10c7-40b7-bd44-3f7902490210 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.367845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.368403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.368682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.368953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.369610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.375975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:39.448759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 769/1540] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:39 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:39.448759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.453968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.484578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 772/1541] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:39.485804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.520457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a53ba-8e28-4ea4-a42b-134cbf0d0729 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.588541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a34d1-ac9a-43bc-8e7c-b7ac8f4271d5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.617048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.617703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.618040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.618245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.620274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.625831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:39.648800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.650950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.652005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.653294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.655197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.659416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:39.709141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 773/1542] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:39 2022] POST /identity/v3/users => generated 282 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:39.711180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.713688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.738495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aae8-766b-444c-906f-eef88ef32ed5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.751249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.751965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.755676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.755676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.821296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.825579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:39.887778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 770/1543] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:39.889295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:39.891673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:39.933414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f98564-cd0c-4ca9-bcda-9af8fbcd2479 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:39.938719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:39.939439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:39.939784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:39.944842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] PATH_INFO: `/v3/projects/3464976e3e024c17ad15685e8af6a061` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:39.946485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3464976e3e024c17ad15685e8af6a061)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:39.948672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.009360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.013204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.077279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 774/1544] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:40.078863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.100286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3464976e3e024c17ad15685e8af6a061', 'request_id': 'req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.112603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3464976e3e024c17ad15685e8af6a061', 'request_id': 'req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.134797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc6a8e-7c83-41cc-90a5-b1d93a946a15 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.135932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3464976e3e024c17ad15685e8af6a061', 'request_id': 'req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.199070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] The token cache is being invalidate because project 3464976e3e024c17ad15685e8af6a061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:40.199070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.199070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.199070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 771/1545] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:39 2022] DELETE /identity/v3/projects/3464976e3e024c17ad15685e8af6a061 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:40.199070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.199600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.223331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:40.266735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.270878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.271303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.271445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.272157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.273785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c325c6-4d00-4723-afe8-66e8b295a3e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.291344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cdf8b-f4a7-4066-824e-2f22c0da2ec2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.293210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 775/1546] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:40.295349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.295349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.295523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.301053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.301785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.302067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.302375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] PATH_INFO: `/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.305178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f2aeebabaf41369e3c6ad40a5f4b41)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.307108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.322063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:40.323949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.337158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 776/1547] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:40.338095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.339579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.351710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ffcab-6d75-48d1-893e-fa0e00165899 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.365858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.367172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.367726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.368127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.369920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.372054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14d1bb-a71d-44b5-8dd0-f9c2cd31d747 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.373952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f2aeebabaf41369e3c6ad40a5f4b41', 'request_id': 'req-db804d04-35d3-409a-a6b0-8568bd376ee6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.382520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f2aeebabaf41369e3c6ad40a5f4b41', 'request_id': 'req-db804d04-35d3-409a-a6b0-8568bd376ee6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.393695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 777/1548] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:40.394581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f2aeebabaf41369e3c6ad40a5f4b41', 'request_id': 'req-db804d04-35d3-409a-a6b0-8568bd376ee6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.395873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9897a0-560c-4e97-bcc3-139d6ff968ad None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.396284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9897a0-560c-4e97-bcc3-139d6ff968ad None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.396954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9897a0-560c-4e97-bcc3-139d6ff968ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.424694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] The token cache is being invalidate because project 08f2aeebabaf41369e3c6ad40a5f4b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:40.425358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.426571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:40.429248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 772/1549] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:40 2022] DELETE /identity/v3/projects/08f2aeebabaf41369e3c6ad40a5f4b41 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:40.430931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.432600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.434996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1a9897a0-560c-4e97-bcc3-139d6ff968ad None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:40.436243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a9897a0-560c-4e97-bcc3-139d6ff968ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.532482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804d04-35d3-409a-a6b0-8568bd376ee6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.537994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.538866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.540251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.540671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] PATH_INFO: `/v3/projects/466b539f668d449c8b9b5f13a8294563/users/abe4d1f397c84738880ba8d23d4c5aaf/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.546932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:40.560283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] RBAC: Authorizing `identity:create_grant(project_id=466b539f668d449c8b9b5f13a8294563, user_id=abe4d1f397c84738880ba8d23d4c5aaf, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.563751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.566767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 778/1550] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:40.568322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade6c6-6af8-455a-865b-e16e5a62833d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.568610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade6c6-6af8-455a-865b-e16e5a62833d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.568907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ade6c6-6af8-455a-865b-e16e5a62833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.587351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 773/1551] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:40 2022] PUT /identity/v3/projects/466b539f668d449c8b9b5f13a8294563/users/abe4d1f397c84738880ba8d23d4c5aaf/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:40.588206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.602262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-d3ade6c6-6af8-455a-865b-e16e5a62833d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:40.603555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3ade6c6-6af8-455a-865b-e16e5a62833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.641783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:40.675980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bda1-6bff-4cfb-af78-732a1588415e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.682668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.682668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.682908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.683096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.683852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:40.684154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.685799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b4a94-c1d5-48a6-b025-5555ee347f2b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.687318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 779/1552] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:40.689154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069353f-8677-4778-9a12-24517575e8cf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.689375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069353f-8677-4778-9a12-24517575e8cf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.689657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069353f-8677-4778-9a12-24517575e8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.698980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 774/1553] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:40.702867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.702867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.702867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.722162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4069353f-8677-4778-9a12-24517575e8cf None None] MFA Rules not processed for user `abe4d1f397c84738880ba8d23d4c5aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:40.723452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4069353f-8677-4778-9a12-24517575e8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.728547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:40.743310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:40.781171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 775/1554] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:40.781171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.802529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d6dd3-a83c-4275-a5fa-9df9d641dff1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.813016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.813651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.814075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.814350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.815000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.817167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.828066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 780/1555] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:40.829242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069353f-8677-4778-9a12-24517575e8cf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.843108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069353f-8677-4778-9a12-24517575e8cf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.854066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.854066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.854066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.854066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.855305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:40.855305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.856305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.864770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 776/1556] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:40.865844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.881622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 781/1557] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:40.882298 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:40.897483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256a9aa-b204-4e09-a7de-cc09fb849db4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.975062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391974bc-9aa4-4d5a-85c5-39d25850e121 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:40.981141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.981803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.982250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.982604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.983362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.985826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:40.987001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:40.987883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:40.988208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:40.988571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:40.989282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:40.991440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.022218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:41.028985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 782/1558] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:41.030016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.030388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 777/1559] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:40 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:41.030872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.032953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.048801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe45e6-5e2e-42d4-9a77-c0f60feeb213 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.106735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ee87c-5540-493c-8f4e-d243b80332ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.107693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.108319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.108593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.108858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.109519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.111460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.111568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.112175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.112454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.112724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] PATH_INFO: `/v3/users/7aeaf5cb88ee4be89ad906f1e1522840` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.113844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aeaf5cb88ee4be89ad906f1e1522840)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.115822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.140103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:41.140369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 783/1560] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:41.144647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.172517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aeaf5cb88ee4be89ad906f1e1522840'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:41.187417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aeaf5cb88ee4be89ad906f1e1522840'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:41.195331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aeaf5cb88ee4be89ad906f1e1522840'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:41.196217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aeaf5cb88ee4be89ad906f1e1522840'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:41.196988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aeaf5cb88ee4be89ad906f1e1522840'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:41.216478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 778/1561] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:41 2022] DELETE /identity/v3/users/7aeaf5cb88ee4be89ad906f1e1522840 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:41.219976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.221544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.229408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836df4be-4e4f-4879-a7a7-0a2faa7b6fe0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.307463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac2623-4805-4e3b-b043-de8a6f2501de admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.310602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.311278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.311586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.311885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.312497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.314437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.325914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.326631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.327214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.327556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.328732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.331411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a13e94-ca5e-4f58-ad81-f5c3a387f68d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.341694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 784/1562] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:41.342714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.344833 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 779/1563] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.346289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.346506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.346634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.346836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.361098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c06b6-1f87-4f54-918d-e697084d6548 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.366089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.366782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.366908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:41.367191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.367526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.368182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.368329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.370620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.376380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 780/1564] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:41.376983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.378245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.390222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f59bfb-f66e-482b-a29f-f47ec0684b7a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.395458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.396078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.396358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.396630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.397288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.398872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 785/1565] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:41.398993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.399973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.408246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:41.412435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 781/1566] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.412435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.412435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.413171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727cae-72a9-4664-b8ab-55de801ab81c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.431500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.431500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.431500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.431500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.431883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.431883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.490825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d581d5f-4e98-4415-b1f3-7c38918da55a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.495054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.496245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.496648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.496763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.507983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.509435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.520639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 782/1567] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:41.523231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.534165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc1984-9603-4643-8f9f-28423c0426f4 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.540009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.540643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.540923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.541197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.541925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.543597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.558556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 783/1568] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.559848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.572078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f47bc-dd99-4f0c-a695-71e96e7d6145 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.577845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 786/1569] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.578744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.579351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.579642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.580991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.587342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] PATH_INFO: `/v3/projects/011b0b1ab8d3450ab1a842825cc8879f/users/8d80c807072a4bda93dfe821c49033cc/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.590793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:41.599856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaba4d5-841c-4a04-997e-d381457cdc32 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.610805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.611536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011b0b1ab8d3450ab1a842825cc8879f, user_id=8d80c807072a4bda93dfe821c49033cc, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.612148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.612713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.613065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.618510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.618660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.618755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.631530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 787/1570] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.632670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.638835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 784/1571] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:41 2022] PUT /identity/v3/projects/011b0b1ab8d3450ab1a842825cc8879f/users/8d80c807072a4bda93dfe821c49033cc/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:41.642494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.643837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00d2b8-4fc9-444f-bdf6-ba1e4eef6e70 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.710960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38fbf-7ce2-49eb-968c-1274f4382255 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.721622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.725046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.725484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.726200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] PATH_INFO: `/v3/projects/20242d235619491d8f622f01699e9f5c/users/a2536bc7c170420da6e107d012555818/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.734747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.735496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.735773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.736046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] PATH_INFO: `/v3/projects/a60e933d4bb44dacbccd876e74e15ee2/users/1f7b093310ef4909b5355c3294d53d27/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.759301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=20242d235619491d8f622f01699e9f5c, user_id=a2536bc7c170420da6e107d012555818, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.767071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a50ef-4679-4060-9a3f-0e4a979dd4bd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.767430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60e933d4bb44dacbccd876e74e15ee2, user_id=1f7b093310ef4909b5355c3294d53d27, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.770614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.791569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 788/1572] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:41 2022] PUT /identity/v3/projects/a60e933d4bb44dacbccd876e74e15ee2/users/1f7b093310ef4909b5355c3294d53d27/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:41.791982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 785/1573] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:41 2022] PUT /identity/v3/projects/20242d235619491d8f622f01699e9f5c/users/a2536bc7c170420da6e107d012555818/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:41.796174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.806460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.807079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.807566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.847334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] MFA Rules not processed for user `a2536bc7c170420da6e107d012555818`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:41.849097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.917518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f7a19-3b47-4b9d-b8f3-dd60fccca4eb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.924388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.925259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.925708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.925984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.926782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.928424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a682533-094f-463a-8653-e6bab23e6064 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.939619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 789/1574] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:41.940951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.941235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.941507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.951012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 786/1575] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:41.951012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:41.951012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:41.965030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8f7a4-f9f1-4cd8-b4ed-c98f5c93a8ec None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:41.969577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:41.970253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:41.970540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:41.970811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] PATH_INFO: `/v3/projects/180ee387dc6541548f5f4d809e54fb3d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:41.971966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180ee387dc6541548f5f4d809e54fb3d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:41.973906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:41.975445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] MFA Rules not processed for user `1f7b093310ef4909b5355c3294d53d27`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:41.976630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.024734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180ee387dc6541548f5f4d809e54fb3d', 'request_id': 'req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.026027 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.027918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.034364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180ee387dc6541548f5f4d809e54fb3d', 'request_id': 'req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.041847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180ee387dc6541548f5f4d809e54fb3d', 'request_id': 'req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.047719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 790/1576] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:42.048597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.057558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] The token cache is being invalidate because project 180ee387dc6541548f5f4d809e54fb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:42.057844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.058540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.059570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c097a-53dc-4d1d-8faa-805da922108b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.060419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 787/1577] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:41 2022] DELETE /identity/v3/projects/180ee387dc6541548f5f4d809e54fb3d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.061051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.062445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.139075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.140183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db23e5-e0cd-47bd-8d2d-0db6ed99eac2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.140394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.140934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.141396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] PATH_INFO: `/v3/projects/011b0b1ab8d3450ab1a842825cc8879f/users/8d80c807072a4bda93dfe821c49033cc/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.143927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011b0b1ab8d3450ab1a842825cc8879f, user_id=8d80c807072a4bda93dfe821c49033cc, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.147181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.147739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.149036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.149660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.149938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.150636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.155905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.162731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 791/1578] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:42 2022] PUT /identity/v3/projects/011b0b1ab8d3450ab1a842825cc8879f/users/8d80c807072a4bda93dfe821c49033cc/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.164546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.166822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.198183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 788/1579] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:42.199758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.202621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.248505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd9bf-f911-4016-83ce-6ac52b3793c4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.275515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967eac3-504e-46b6-ba86-fb9f685459c3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.280734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.281837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.282316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.283243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] PATH_INFO: `/v3/users/4baf32d185fb4396af16dc42b97712b0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.284879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4baf32d185fb4396af16dc42b97712b0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.287510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.319901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.320848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.320982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.321768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.322581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.324915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49752f-241f-4205-b152-42b5d905a810 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.348401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4baf32d185fb4396af16dc42b97712b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.365202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4baf32d185fb4396af16dc42b97712b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.366583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.368518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 792/1580] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:42.370007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.370614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.371173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.381351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4baf32d185fb4396af16dc42b97712b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.382360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4baf32d185fb4396af16dc42b97712b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.384000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4baf32d185fb4396af16dc42b97712b0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:42.407110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 789/1581] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:42 2022] DELETE /identity/v3/users/4baf32d185fb4396af16dc42b97712b0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.407110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.407533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.416862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] MFA Rules not processed for user `8d80c807072a4bda93dfe821c49033cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:42.418923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.428105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.488431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a8e5-54ec-43c1-bf1d-9ce913315fc6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.493937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.495198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.495809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.496341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.497277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.499877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.535761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 790/1582] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:42.538320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.541329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.544073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 793/1583] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:42.545234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.558654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca82ea-32df-4fef-82c2-0169b439d1dd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.559540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ab234-f37a-45c4-95ab-79e20908a86a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.566164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.567099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.567518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.568311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.568822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.569173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.570103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.570712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.571033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.571254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.571881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.574854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.582213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 794/1584] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:42.583018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.587182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.608898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 791/1585] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/users => generated 291 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:42.608898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.623058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affda12-ef8b-4345-8e7f-ffa658cd4531 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.628514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.629331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.630165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.630165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] PATH_INFO: `/v3/projects/174bff59f4bd4c018f6ab99c188552ca/users/891505a969844c6881c0d4598ec6f1a3/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.648624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=174bff59f4bd4c018f6ab99c188552ca, user_id=891505a969844c6881c0d4598ec6f1a3, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.652503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.667040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25923e-b3d8-49bd-bf24-f3298841c9dd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.671070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 792/1586] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:42 2022] PUT /identity/v3/projects/174bff59f4bd4c018f6ab99c188552ca/users/891505a969844c6881c0d4598ec6f1a3/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.671960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.744967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.745657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.746210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.746487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.752941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6affa-bd50-4603-b2d9-a00633de0aeb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.757303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.758078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.758377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.758649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.759801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.761867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.775679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 793/1587] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:42.776690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.786202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467a179-26e7-42d6-b47f-9e374579d64a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.790890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.791539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.791826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.792093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.792729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.794313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.808521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 794/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:42.809506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.820645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf212b0-0c03-4d7e-b9a1-90094bb97cc5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.823778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.825370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.825966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.826631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.826909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.827202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] PATH_INFO: `/v3/projects/9b911c5061f24f149bd1dde90fb9de56/users/d9fa554fdd9743b0b96e4884e03b8dca/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.830054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.839342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 795/1589] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:42.840314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.853860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a5523-2b31-4dd1-b4d7-49dfc9a9f371 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:42.855011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=8,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:42.859088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b911c5061f24f149bd1dde90fb9de56, user_id=d9fa554fdd9743b0b96e4884e03b8dca, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.859246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:42.859787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.860065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.860417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] PATH_INFO: `/v3/projects/174bff59f4bd4c018f6ab99c188552ca/users/891505a969844c6881c0d4598ec6f1a3/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.862334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174bff59f4bd4c018f6ab99c188552ca, user_id=891505a969844c6881c0d4598ec6f1a3, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:42.863453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528c785-ad58-44cb-9f57-3d9800138cd3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.865088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:42.885641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 796/1590] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:42 2022] PUT /identity/v3/projects/174bff59f4bd4c018f6ab99c188552ca/users/891505a969844c6881c0d4598ec6f1a3/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.886535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:42.888250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.896887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 795/1591] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:42 2022] PUT /identity/v3/projects/9b911c5061f24f149bd1dde90fb9de56/users/d9fa554fdd9743b0b96e4884e03b8dca/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:42.900003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:42.900003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:42.900003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:42.941826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] MFA Rules not processed for user `891505a969844c6881c0d4598ec6f1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:42.943254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:42.986582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f4f8-8363-4051-b1c0-8f6d4fa2c341 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.052557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 796/1592] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:43.056752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.068331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.069214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.069561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.069831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] PATH_INFO: `/v3/projects/30a568656cb840fab636552c55776d94` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.071239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a568656cb840fab636552c55776d94)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.073916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.139473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adad77-c0c7-4c4b-9969-de160b6dca6e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.140076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a568656cb840fab636552c55776d94', 'request_id': 'req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.147082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a568656cb840fab636552c55776d94', 'request_id': 'req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.158626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a568656cb840fab636552c55776d94', 'request_id': 'req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.172768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] The token cache is being invalidate because project 30a568656cb840fab636552c55776d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:43.173398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.174121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.179534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 797/1593] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:42 2022] DELETE /identity/v3/projects/30a568656cb840fab636552c55776d94 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:43.180440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.182407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.215546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:43.217819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.218945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.219561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.219964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.220943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.224203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9361fd-8423-405c-ba11-db5983a31b05 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.239012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 797/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:43.240588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.240943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.241277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.268379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df5b6-db81-4bbf-ae9d-af6dd710a45e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.277399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.278190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.278559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.278754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:43.279320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.280089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.281253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.297720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 798/1595] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:43.299250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.302165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.319330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21567a-9760-4ae2-832c-c2d85a0fb172 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.322602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.323494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.323791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.324065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] PATH_INFO: `/v3/projects/9b911c5061f24f149bd1dde90fb9de56/users/d9fa554fdd9743b0b96e4884e03b8dca/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.326039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b911c5061f24f149bd1dde90fb9de56, user_id=d9fa554fdd9743b0b96e4884e03b8dca, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.328715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe353c-1410-4fa9-9f09-7a17825056e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.335946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:43.342973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 799/1596] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:43 2022] PUT /identity/v3/projects/9b911c5061f24f149bd1dde90fb9de56/users/d9fa554fdd9743b0b96e4884e03b8dca/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:43.344504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.344794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.345180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.368235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:43.369491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.371604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.373074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e7fc3-7ac0-4a33-9f20-6601206f80fe service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.382697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 798/1597] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:43.388688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.389115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.389393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.430227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] MFA Rules not processed for user `d9fa554fdd9743b0b96e4884e03b8dca`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:43.431847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.455523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 800/1598] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:43.460262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.474790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8a4cf-401f-4d9b-915f-0788f99b818a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.489390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.490026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.490304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.490782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.492733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:43.493037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.494746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.518024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 801/1599] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:43.520215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.522874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.542430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3a56b-dd58-4c34-ba98-e2968c5da5cb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.548630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.549910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.549910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.549910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] PATH_INFO: `/v3/users/abe4d1f397c84738880ba8d23d4c5aaf` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.551429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe4d1f397c84738880ba8d23d4c5aaf)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.554285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.564171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 799/1600] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:43.565371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.595797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c391b2-575b-467d-afab-68d9f480c063 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.610172 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.612422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.613962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.614330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.615086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.620581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.654113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe4d1f397c84738880ba8d23d4c5aaf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.660657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 800/1601] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:43.664951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.667088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.674572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe4d1f397c84738880ba8d23d4c5aaf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.683808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe4d1f397c84738880ba8d23d4c5aaf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.685372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe4d1f397c84738880ba8d23d4c5aaf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.686006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe4d1f397c84738880ba8d23d4c5aaf'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:43.704312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 802/1602] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:43 2022] DELETE /identity/v3/users/abe4d1f397c84738880ba8d23d4c5aaf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:43.705428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.707244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.747975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:43.773850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4c02a-66d5-4640-837b-f56912d4ca3a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.847010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd078-e43b-44c1-a5f7-692a85671939 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.859526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.860171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.860466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.860721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.861335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.863406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363f0b5-4000-46fe-b5cc-5ce7552d0036 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.864619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.866310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.866722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.867278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.870648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.872466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:43.894537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 803/1603] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:43.896442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.896982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.897264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.924676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:43.925813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.934101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 804/1604] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:43.934935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.944578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd033e1e-1122-49e5-b4d3-625b66a28d37 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:43.948051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.949521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.951367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:43.952105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:43.952472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:43.952847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:43.953596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:43.955463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:43.968110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 805/1605] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:43.983186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:43.983186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:43.983503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 801/1606] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:43.983503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:44.002150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc873fb8-e48f-40bc-ac16-cd837665723e service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:44.009658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:44.012692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.012692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.012692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.012692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:44.014150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:44.031278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 802/1607] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:44.035460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:44.056488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cdfd0-42ee-40e2-b3fc-2359fd70a0bf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:44.058355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3814440-71df-4f08-b43c-a38222bc6808 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:44.060678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:44.061517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.061840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.062362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.066951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:44.067736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.068246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.068614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] PATH_INFO: `/v3/projects/11fcf369aff04afca1e501f8714117af/users/2774a77cb0e54f64b854d0aff9c7d270/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.074296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:44.075975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66197a59-36fb-44ff-8c17-6409d67d9e5a service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:44.087602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fcf369aff04afca1e501f8714117af, user_id=2774a77cb0e54f64b854d0aff9c7d270, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:44.089742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 806/1608] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:44.091189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:44.111098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 803/1609] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:44 2022] PUT /identity/v3/projects/11fcf369aff04afca1e501f8714117af/users/2774a77cb0e54f64b854d0aff9c7d270/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:44.116134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.116359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.116637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.139742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:44.141457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:44.230812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 807/1610] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:44.260306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:44.425483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f9b7-dd23-4de1-aca6-16b1a19b8861 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:44.436144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:44.437743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.438267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.438947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.452390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:44.454576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:44.468250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 804/1611] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:44.680211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:44.681842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:44.809654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e4ad-40b1-4c49-ab55-c86cda0a8099 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:44.816660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:44.817432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:44.818880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:44.819624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:44.823913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:44.873019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:44.923361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:44.924897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:44.942861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 808/1612] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:45.662135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:45.664171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:45.686963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290860c-697f-4bd4-a739-7e5b78e668dd service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:45.693274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:45.694951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:45.695215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:45.695625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] PATH_INFO: `/v3/projects/466b539f668d449c8b9b5f13a8294563` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:45.698865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=466b539f668d449c8b9b5f13a8294563)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:45.703698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:45.721781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:45.783321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466b539f668d449c8b9b5f13a8294563', 'request_id': 'req-31261bea-17d7-4493-ad90-5a98010f5c3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:45.790671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466b539f668d449c8b9b5f13a8294563', 'request_id': 'req-31261bea-17d7-4493-ad90-5a98010f5c3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:45.803552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466b539f668d449c8b9b5f13a8294563', 'request_id': 'req-31261bea-17d7-4493-ad90-5a98010f5c3c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:45.817501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] The token cache is being invalidate because project 466b539f668d449c8b9b5f13a8294563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:45.817877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:45.818595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-31261bea-17d7-4493-ad90-5a98010f5c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:45.820572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 805/1613] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:45 2022] DELETE /identity/v3/projects/466b539f668d449c8b9b5f13a8294563 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:45.825051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:45.841628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2c2f-cb07-4cb4-bb22-f96879ea24a3 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:45.940485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:45.941687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:45.942132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:45.942682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] PATH_INFO: `/v3/users/8d80c807072a4bda93dfe821c49033cc` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:45.944575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d80c807072a4bda93dfe821c49033cc)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:45.948021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.016656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d80c807072a4bda93dfe821c49033cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.035558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d80c807072a4bda93dfe821c49033cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.043400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d80c807072a4bda93dfe821c49033cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.044272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d80c807072a4bda93dfe821c49033cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.044865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d80c807072a4bda93dfe821c49033cc'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.060227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 809/1614] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:45 2022] DELETE /identity/v3/users/8d80c807072a4bda93dfe821c49033cc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:46.270451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.270735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.271037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.320079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] MFA Rules not processed for user `2774a77cb0e54f64b854d0aff9c7d270`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:46.321327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.385605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.390032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.489822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 806/1615] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:46.493899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:46.494800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.496665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.525866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18d5f1-2010-4897-8e23-4c414333016e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.526780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe7e53-e55a-49d4-8d4b-4e480071f41a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.534921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.535834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.536115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.536384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.537158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.539326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.539628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.540733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.541147 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.541753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] PATH_INFO: `/v3/projects/011b0b1ab8d3450ab1a842825cc8879f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.543935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011b0b1ab8d3450ab1a842825cc8879f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.547059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.573989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 807/1616] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:46.575266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.578131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.617106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011b0b1ab8d3450ab1a842825cc8879f', 'request_id': 'req-980781cb-4a0e-43b5-9423-817a42bfac24'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.638712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011b0b1ab8d3450ab1a842825cc8879f', 'request_id': 'req-980781cb-4a0e-43b5-9423-817a42bfac24'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.655184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011b0b1ab8d3450ab1a842825cc8879f', 'request_id': 'req-980781cb-4a0e-43b5-9423-817a42bfac24'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.668792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] The token cache is being invalidate because project 011b0b1ab8d3450ab1a842825cc8879f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:46.669185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.669881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.671783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 810/1617] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:46 2022] DELETE /identity/v3/projects/011b0b1ab8d3450ab1a842825cc8879f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:46.672663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.674109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.675411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab488a97-3666-43c3-84dc-4a02ab85353b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.687173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:46.761466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.762306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.762590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.762864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] PATH_INFO: `/v3/users/1f7b093310ef4909b5355c3294d53d27` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.763825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980781cb-4a0e-43b5-9423-817a42bfac24 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.764074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7b093310ef4909b5355c3294d53d27)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.766088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.771318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.772785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.773069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.773339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.773979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.775971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53d880-16e9-4d16-a905-a0cd59f87c37 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.802425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 811/1618] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:46.804000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.804359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.805252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.825168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7b093310ef4909b5355c3294d53d27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.825818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:46.826898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.835846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 812/1619] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:46.836796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.846538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd292bd-fb37-44c6-b1f4-25fac8079a23 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.852543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.852543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.852543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.852543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.853062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.854732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf3fa0-ef05-4f09-af67-601b5a6641c5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.855552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7b093310ef4909b5355c3294d53d27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.856442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7b093310ef4909b5355c3294d53d27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.863630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7b093310ef4909b5355c3294d53d27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.864315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7b093310ef4909b5355c3294d53d27'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:46.865868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 813/1620] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:46.867054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e410d4-96b3-43f7-b5d4-9e154ae6e07f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.867435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e410d4-96b3-43f7-b5d4-9e154ae6e07f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.867709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e410d4-96b3-43f7-b5d4-9e154ae6e07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.879065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 808/1621] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:46 2022] DELETE /identity/v3/users/1f7b093310ef4909b5355c3294d53d27 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:46.880040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.882817 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.889481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-30e410d4-96b3-43f7-b5d4-9e154ae6e07f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:46.890734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-30e410d4-96b3-43f7-b5d4-9e154ae6e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.936390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:46.947000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab500ca1-ae83-4674-802c-82eb7f142800 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.951963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:46.952213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 814/1622] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:46.952639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.952919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.953190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] PATH_INFO: `/v3/projects/a156c390c144446280de6920e45ca855/users/935e7449ff544b2db80b7837d702f470/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.953531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:46.953812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:46.954081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:46.970198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a156c390c144446280de6920e45ca855, user_id=935e7449ff544b2db80b7837d702f470, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:46.972809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:46.972944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:46.974138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:46.987192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 815/1623] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:46.987192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:46.995957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f9b1b-b5d7-484b-8821-125a90911051 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:46.997730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 809/1624] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:46 2022] PUT /identity/v3/projects/a156c390c144446280de6920e45ca855/users/935e7449ff544b2db80b7837d702f470/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:46.998669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.000242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.014432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:47.056320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.056925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.057232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.057511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.058275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:47.058581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.060218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.060357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b063a5-abbf-4a62-8486-0a1bc5cb7ac7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.063911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.064522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.064797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.065061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.071554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 816/1625] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:47.072348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.077794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.079398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.082023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a98af-7731-4a14-8e6e-59dd65d025e6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.086591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.087207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.087492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.087775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.088486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.089664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 810/1626] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:47.090048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.090339 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.099791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff799d8-2b9c-4cb7-8061-e0ef8e5df8c9 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.101129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 817/1627] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:47.105372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.107072 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.107785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.108135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.108473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.109175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.111165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.115681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c33bf-1563-454c-83c8-5d27a30da8c0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.121624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.121624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.121874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.122022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] PATH_INFO: `/v3/projects/a156c390c144446280de6920e45ca855/users/935e7449ff544b2db80b7837d702f470/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.123996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a156c390c144446280de6920e45ca855, user_id=935e7449ff544b2db80b7837d702f470, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.126642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.138168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 818/1628] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:47 2022] PUT /identity/v3/projects/a156c390c144446280de6920e45ca855/users/935e7449ff544b2db80b7837d702f470/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:47.144144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 811/1629] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:47 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:47.154264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.233774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be92c0-15fd-41d5-b1df-0805a4d2a1ef admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.296707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.297328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.297615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.297869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.298499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.300573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.310540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:47.324019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 819/1630] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:47 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:47.328247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.340216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a1676-5500-46bb-a300-8c9e51fe7545 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.345159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.345835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.346403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.346505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.347442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.349282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.360954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 812/1631] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:47.365182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.376134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52fd04-834d-48c4-a426-b2ecedb00a7a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.380704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.381305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.381581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.381845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.382504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.384149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecb569-4c4b-4604-acef-d41b52feb349 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.398905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 820/1632] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:47.400288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.414128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf077c5-d3f4-4185-a551-cdf314b89242 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.419800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.420978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.421255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.421556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] PATH_INFO: `/v3/projects/75b8126438734e5eb8e0181ac4b19151/users/f630238e5e1a410aa4e172c60c304d95/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.438702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b8126438734e5eb8e0181ac4b19151, user_id=f630238e5e1a410aa4e172c60c304d95, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.441417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.457642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 813/1633] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:47 2022] PUT /identity/v3/projects/75b8126438734e5eb8e0181ac4b19151/users/f630238e5e1a410aa4e172c60c304d95/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:47.468256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.468538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.468821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.506485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] MFA Rules not processed for user `f630238e5e1a410aa4e172c60c304d95`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:47.507782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.520867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.522857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.595558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 821/1634] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:47.600674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.615450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc134a-cf29-4081-b221-49f6b4086289 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.618268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.618923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.619232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.619529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] PATH_INFO: `/v3/projects/a60e933d4bb44dacbccd876e74e15ee2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.620784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60e933d4bb44dacbccd876e74e15ee2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.622979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.628822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ec966-c73e-4b6d-9eaa-595b7c55f0fc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.634564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.635515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.635818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.636088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.636752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.638660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.678089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 822/1635] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:47 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:47.679528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.686399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60e933d4bb44dacbccd876e74e15ee2', 'request_id': 'req-807bb6b3-ef1f-4381-88bc-47dea0d126e5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.696377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60e933d4bb44dacbccd876e74e15ee2', 'request_id': 'req-807bb6b3-ef1f-4381-88bc-47dea0d126e5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.709445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60e933d4bb44dacbccd876e74e15ee2', 'request_id': 'req-807bb6b3-ef1f-4381-88bc-47dea0d126e5'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.711292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:47.725594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] The token cache is being invalidate because project a60e933d4bb44dacbccd876e74e15ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:47.726097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.727071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.732083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 814/1636] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:47 2022] DELETE /identity/v3/projects/a60e933d4bb44dacbccd876e74e15ee2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:47.733314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.735119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.766447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccbb7d-4306-4548-8a41-5dc72929f355 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.772208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.772820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.773099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.773364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.773989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.777052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.811623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 823/1637] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:47 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:47.815628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.817510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.830768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bb6b3-ef1f-4381-88bc-47dea0d126e5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.831205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90777915-cea5-485a-98f6-db49dca87986 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.837326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.838160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.838633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.838871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.839105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] PATH_INFO: `/v3/users/d9fa554fdd9743b0b96e4884e03b8dca` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.839874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.840291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.840750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.841115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fa554fdd9743b0b96e4884e03b8dca)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.841905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.843327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.843836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.855250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 824/1638] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:47.859291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.876581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c52c1-6562-409d-a1ca-8f73559b8166 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:47.882774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:47.883435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:47.884014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:47.884014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] PATH_INFO: `/v3/projects/c84cf83c23ca4909b673f2d0fc35f072/users/dfcef57b5f9f43d0a1d154db1d0adacb/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:47.903556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84cf83c23ca4909b673f2d0fc35f072, user_id=dfcef57b5f9f43d0a1d154db1d0adacb, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:47.907637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:47.915098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fa554fdd9743b0b96e4884e03b8dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.932808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 825/1639] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:47 2022] PUT /identity/v3/projects/c84cf83c23ca4909b673f2d0fc35f072/users/dfcef57b5f9f43d0a1d154db1d0adacb/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:47.937337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:47.937635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fa554fdd9743b0b96e4884e03b8dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.938386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fa554fdd9743b0b96e4884e03b8dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.940002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:47.946523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fa554fdd9743b0b96e4884e03b8dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.947455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:47.948719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fa554fdd9743b0b96e4884e03b8dca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:47.968683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 815/1640] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:47 2022] DELETE /identity/v3/users/d9fa554fdd9743b0b96e4884e03b8dca => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:48.027803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8b420-fd97-48f4-b55f-d76411efb8af admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.102609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.103311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.103655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.104031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.104754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.106434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591019f-93ae-4a59-8446-4abfd07a1106 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:48.122255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 826/1641] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:48.123864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:48.137327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b878d-7a5b-41a0-90d2-30512852a681 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.143998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.145075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.145249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.145625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] PATH_INFO: `/v3/projects/c84cf83c23ca4909b673f2d0fc35f072/users/dfcef57b5f9f43d0a1d154db1d0adacb/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.147952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84cf83c23ca4909b673f2d0fc35f072, user_id=dfcef57b5f9f43d0a1d154db1d0adacb, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.151175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c864-8d56-4003-bed4-91422418d75f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:48.163392 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 816/1642] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:48 2022] PUT /identity/v3/projects/c84cf83c23ca4909b673f2d0fc35f072/users/dfcef57b5f9f43d0a1d154db1d0adacb/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:48.176816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.177625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.177902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.208245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] MFA Rules not processed for user `dfcef57b5f9f43d0a1d154db1d0adacb`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:48.209355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.231739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.232421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.232965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.283200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] MFA Rules not processed for user `935e7449ff544b2db80b7837d702f470`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:48.284832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.329680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 827/1643] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:48.342291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:48.345210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.353240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:48.411073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 817/1644] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:48.428972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:48.440956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:48.442563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.464359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c6c07-5334-4983-a0cf-27728f8addf0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.468461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.469137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.469468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.469762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.486576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.488267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:48.499855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 828/1645] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:48.512610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:48.514158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.591818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee11046-e342-4ec9-96fa-5dad55878470 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.600435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.601105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.601378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.601639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.608543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.679582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8843d5-af4d-46a9-8de5-889fe0db4b6e service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.692701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.693373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.693663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.693960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] PATH_INFO: `/v3/projects/9b911c5061f24f149bd1dde90fb9de56` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.695397 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b911c5061f24f149bd1dde90fb9de56)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.699444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:48.768962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.771745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b911c5061f24f149bd1dde90fb9de56', 'request_id': 'req-995e35a0-5d1c-4877-8e6e-96c6a49d0453'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:48.779888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b911c5061f24f149bd1dde90fb9de56', 'request_id': 'req-995e35a0-5d1c-4877-8e6e-96c6a49d0453'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:48.782756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:48.792694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b911c5061f24f149bd1dde90fb9de56', 'request_id': 'req-995e35a0-5d1c-4877-8e6e-96c6a49d0453'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:48.798634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 818/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:48.799824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:48.817041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474df9b-e10d-410f-b1c4-569a1532c314 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:48.827984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] The token cache is being invalidate because project 9b911c5061f24f149bd1dde90fb9de56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:48.829625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:48.831273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-995e35a0-5d1c-4877-8e6e-96c6a49d0453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:48.834283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 829/1647] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:48 2022] DELETE /identity/v3/projects/9b911c5061f24f149bd1dde90fb9de56 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:48.840526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.840818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.841103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.900353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:48.902344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:48.944818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:48.945674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:48.946044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:48.946468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] PATH_INFO: `/v3/users/f630238e5e1a410aa4e172c60c304d95` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:48.947883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f630238e5e1a410aa4e172c60c304d95)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:48.949952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.041793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:49.058290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f630238e5e1a410aa4e172c60c304d95'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:49.069258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 830/1648] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:49.070211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.071613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:49.075056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f630238e5e1a410aa4e172c60c304d95'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:49.075966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f630238e5e1a410aa4e172c60c304d95'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:49.082680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:49.090479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f630238e5e1a410aa4e172c60c304d95'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:49.091377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e6fe21d7-b550-4553-a797-0762d1d3856f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f630238e5e1a410aa4e172c60c304d95'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:49.136483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 819/1649] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:48 2022] DELETE /identity/v3/users/f630238e5e1a410aa4e172c60c304d95 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:49.140011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf83a3-a7fb-4d11-ba62-913538faabfc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.140815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf83a3-a7fb-4d11-ba62-913538faabfc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.142954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf83a3-a7fb-4d11-ba62-913538faabfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.196579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350cbf-9a38-49cb-874d-35d819fc2695 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.202637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-75cf83a3-a7fb-4d11-ba62-913538faabfc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:49.204027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-75cf83a3-a7fb-4d11-ba62-913538faabfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:49.286553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.288670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.288953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.290100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.293941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:49.325440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 820/1650] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:49.327132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.327514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.327897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.376848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:49.380316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:49.406039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 821/1651] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:49.408039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.439083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3d6a5-2bf5-48b1-9ca9-a1a63921422d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.454187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.458132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.458435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.459844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.462471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:49.462728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.462867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.466484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.468297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.491029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 831/1652] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:49.499968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 822/1653] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:49.509633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.510952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.517847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:49.537921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a173f-31c4-4b9b-a9e4-d6fd57630125 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.551198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.551835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.552113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.552389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.553422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.555268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.606943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 823/1654] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:49 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:49.617863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.650646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6966ed-a5c6-4d9d-a9f2-7102ecfeec3c service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.689502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:49.748998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6155-3383-4c92-a23a-3aa9c744dcfa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.752319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.754176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.754560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.754806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.755142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.755277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.755470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.756170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.756531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.758583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.777303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.779666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.795977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 824/1655] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:49 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:49.802843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.803699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 832/1656] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:49.808859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:49.814313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc91bb5-b7a9-4140-8d89-76941135cba1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.820570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.821428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.822056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.822735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.824030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.828566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.845772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 825/1657] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:49.849588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.862675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385e8a9-c417-4b68-8129-0453eb48d71d service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.869344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.869916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.870390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.870593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.873798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.876454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0928bda-d676-4fd4-b799-05b82c5f4f40 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.895274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 833/1658] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:49.902102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:49.917121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828da8-35b8-483c-be20-4d4523c5bb4b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:49.925165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:49.926278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.926666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.927058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] PATH_INFO: `/v3/projects/dcd5d2e356d34467912289d176b4923c/users/5f3aea8d4efd42e98f3d645ce0c0a076/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:49.957737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd5d2e356d34467912289d176b4923c, user_id=5f3aea8d4efd42e98f3d645ce0c0a076, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:49.961201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:49.979210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 826/1659] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:49 2022] PUT /identity/v3/projects/dcd5d2e356d34467912289d176b4923c/users/5f3aea8d4efd42e98f3d645ce0c0a076/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:49.989571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:49.989876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:49.990135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.023441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] MFA Rules not processed for user `5f3aea8d4efd42e98f3d645ce0c0a076`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:50.025034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.103527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 834/1660] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:50.108063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.158004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.159735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.187501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53d03a-9e22-4751-8bbe-5a5d67476b5a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.193108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.193914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.194196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.194626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.195365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.197292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.227167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 827/1661] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:50 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:50.231174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.246837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2155a8-e212-4b24-8be4-66012bb31af6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.297406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:50.299200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5db626-61c3-42f9-9627-f946a89cd7d1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.305418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.305596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.306044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.306152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.306320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.306411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.306596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] PATH_INFO: `/v3/projects/75b8126438734e5eb8e0181ac4b19151` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.306973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.307703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.307878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b8126438734e5eb8e0181ac4b19151)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.309885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.310563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.335776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 828/1662] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:50 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:50.339617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.350080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f7bd-6077-428c-8612-605231d5d91e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.355342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.355983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.356258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.356642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.357363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.359537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.372033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 829/1663] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:50.373522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b8126438734e5eb8e0181ac4b19151', 'request_id': 'req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.377438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.382014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b8126438734e5eb8e0181ac4b19151', 'request_id': 'req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.389474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650eb9-31ad-44fb-8d16-d58958de74fd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.390945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b8126438734e5eb8e0181ac4b19151', 'request_id': 'req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.403815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] The token cache is being invalidate because project 75b8126438734e5eb8e0181ac4b19151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:50.404095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.404757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.409142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 835/1664] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:50 2022] DELETE /identity/v3/projects/75b8126438734e5eb8e0181ac4b19151 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:50.410399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.412048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.427119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:50.467305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.467305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.467305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.467305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] PATH_INFO: `/v3/projects/2a818a879cd34f93967de4f6b7527cd3/users/ae019b6073a3432fa203b78300a6aa6a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.478504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a818a879cd34f93967de4f6b7527cd3, user_id=ae019b6073a3432fa203b78300a6aa6a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.481294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.488268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a5bce-96c3-4bc3-b30e-21404e620c2c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.493288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.493929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.494206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.494476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] PATH_INFO: `/v3/users/dfcef57b5f9f43d0a1d154db1d0adacb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.495305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 830/1665] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:50 2022] PUT /identity/v3/projects/2a818a879cd34f93967de4f6b7527cd3/users/ae019b6073a3432fa203b78300a6aa6a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:50.495690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfcef57b5f9f43d0a1d154db1d0adacb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.497672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.499322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.500843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.548419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfcef57b5f9f43d0a1d154db1d0adacb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.561381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfcef57b5f9f43d0a1d154db1d0adacb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.569077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfcef57b5f9f43d0a1d154db1d0adacb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.570018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfcef57b5f9f43d0a1d154db1d0adacb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.570773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfcef57b5f9f43d0a1d154db1d0adacb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:50.571804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9483de-71a9-4e9f-b743-1fe58feb6f4a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.578584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.579249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.579648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.579919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.580567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.582531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a61ec5-6fed-496d-971f-f5633fc15962 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.587243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 836/1666] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:50 2022] DELETE /identity/v3/users/dfcef57b5f9f43d0a1d154db1d0adacb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:50.592825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 831/1667] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:50.597115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.598713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.605536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:50.674855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b56d7-a144-4a3f-84d4-ffd0d00609f1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.681317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.681693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.681975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.682272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] PATH_INFO: `/v3/projects/2a818a879cd34f93967de4f6b7527cd3/users/ae019b6073a3432fa203b78300a6aa6a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.684648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a818a879cd34f93967de4f6b7527cd3, user_id=ae019b6073a3432fa203b78300a6aa6a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.687070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.699841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 837/1668] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:50 2022] PUT /identity/v3/projects/2a818a879cd34f93967de4f6b7527cd3/users/ae019b6073a3432fa203b78300a6aa6a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:50.712706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.712992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.713290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.746161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] MFA Rules not processed for user `ae019b6073a3432fa203b78300a6aa6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:50.747422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.786851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.788842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.851353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 832/1669] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:50.860546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c749414-e43f-4217-820a-25f9423ed659 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.862395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.864196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.864988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.865675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.866016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.866340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.868187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:50.879189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ce73a-c957-4292-9c53-c282dc977d29 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:50.884017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:50.884949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:50.885116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:50.885355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:50.897488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.899020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.915677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 833/1670] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:50.949115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:50.951433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:50.967057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 838/1671] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:50.989029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:50.990542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.044983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:51.094277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49444ea-0ee7-4b50-a6fa-319d8ae40b6f service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.099191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.099848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.100120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.100386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] PATH_INFO: `/v3/projects/c84cf83c23ca4909b673f2d0fc35f072` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.101568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84cf83c23ca4909b673f2d0fc35f072)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.103811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.155343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.176657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d094ddd-6316-4c0c-8188-07cb7ee70603 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.177740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84cf83c23ca4909b673f2d0fc35f072', 'request_id': 'req-1568aa58-a319-42cc-b8f5-7132e6f90861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.185448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84cf83c23ca4909b673f2d0fc35f072', 'request_id': 'req-1568aa58-a319-42cc-b8f5-7132e6f90861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.199416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84cf83c23ca4909b673f2d0fc35f072', 'request_id': 'req-1568aa58-a319-42cc-b8f5-7132e6f90861'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.217000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] The token cache is being invalidate because project c84cf83c23ca4909b673f2d0fc35f072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:51.217596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.218451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1568aa58-a319-42cc-b8f5-7132e6f90861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.225834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 834/1672] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:50 2022] DELETE /identity/v3/projects/c84cf83c23ca4909b673f2d0fc35f072 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:51.227603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cf801-59ac-4bee-a602-de304e9d92fa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.228093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cf801-59ac-4bee-a602-de304e9d92fa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.228657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cf801-59ac-4bee-a602-de304e9d92fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.239871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:51.254907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb4cf801-59ac-4bee-a602-de304e9d92fa None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:51.256496 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb4cf801-59ac-4bee-a602-de304e9d92fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.261056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.261952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.262390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.262728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] PATH_INFO: `/v3/users/5f3aea8d4efd42e98f3d645ce0c0a076` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.264045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f3aea8d4efd42e98f3d645ce0c0a076)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.266360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.267872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 835/1673] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:51.275794 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.276217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.276387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.298958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:51.300637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.310446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 836/1674] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:51.314895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.326160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2634ed-58ba-4322-9966-9299ae2faf40 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.328646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f3aea8d4efd42e98f3d645ce0c0a076'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.331321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.331671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.332431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.332701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.333353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:51.333652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.335284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.344842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f3aea8d4efd42e98f3d645ce0c0a076'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.348489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 837/1675] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:24:51.352685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.356472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f3aea8d4efd42e98f3d645ce0c0a076'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.357876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f3aea8d4efd42e98f3d645ce0c0a076'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.358610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-94a3b531-3d71-4fb0-96c2-ee37bef532a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f3aea8d4efd42e98f3d645ce0c0a076'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.365887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595509-4ed2-43ad-bd7b-5638b9142ab7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.371212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.371821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.372095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.372359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.373112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.375068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.375687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 839/1676] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:51 2022] DELETE /identity/v3/users/5f3aea8d4efd42e98f3d645ce0c0a076 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:51.380022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.380294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.380572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.402948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:51.404434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.406102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 838/1677] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:51.410242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.412336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.473456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 840/1678] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:51.484382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.486851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.495936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b24cc-7d12-4669-b007-c8e0da094b84 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.502884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.503639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.503917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.504184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.504829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.506818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.522481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:51.543778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 839/1679] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/users => generated 305 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:51.545915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.558814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce50a-03f3-4999-a6c7-dec2dbc9abc0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.565599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.565992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6019ec-6b9a-4a3c-b632-c614451c8369 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.566231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.566507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.566773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.567479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.569139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.570887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.571554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.571832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.572469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.582444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 840/1680] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:51.584048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.585918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.587408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.590057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.597601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 841/1681] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:51.598960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.605098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ef088-f51a-4c93-ab65-d76261112682 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.608660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e1b7c-7def-4daa-9ec2-8ef27bef31ef service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.621250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.621918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.622028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.622197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.622885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.623009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] PATH_INFO: `/v3/users/8b3d2738dc60481c963bc18ce3dcd2bf` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.624016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.624016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.624342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.625981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3d2738dc60481c963bc18ce3dcd2bf)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.626184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.628009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.633659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:51.642113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 842/1682] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:51.646284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.661632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fa17f-6827-4ebe-94cd-f36a362f585b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.670043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.671118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.671665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.672126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] PATH_INFO: `/v3/projects/f7c1554d89c54eef84ff326e6b2b198a/users/8b15ee6e86e44c059320b2f765683ef7/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.691536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c1554d89c54eef84ff326e6b2b198a, user_id=8b15ee6e86e44c059320b2f765683ef7, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.694704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3627c-cccd-474b-b09f-a0adf908e67c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.703425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3d2738dc60481c963bc18ce3dcd2bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.711678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 843/1683] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:51 2022] PUT /identity/v3/projects/f7c1554d89c54eef84ff326e6b2b198a/users/8b15ee6e86e44c059320b2f765683ef7/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:51.722921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3d2738dc60481c963bc18ce3dcd2bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.723830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3d2738dc60481c963bc18ce3dcd2bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.725039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.725380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.725689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.732808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3d2738dc60481c963bc18ce3dcd2bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.733554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-746a9611-db86-4805-b2e9-08faccb62917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3d2738dc60481c963bc18ce3dcd2bf'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:51.755754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 841/1684] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:51 2022] DELETE /identity/v3/users/8b3d2738dc60481c963bc18ce3dcd2bf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:51.761483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.761913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.762331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.764052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] MFA Rules not processed for user `8b15ee6e86e44c059320b2f765683ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:51.765333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.790411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:51.791814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.869434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 844/1685] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:51.873942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.874529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 842/1686] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:51.880743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.886387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.887788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.896068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03becc8a-5d52-4807-a2dc-7a55e4c6fcd3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.904197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:51.905213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:51.905621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:51.906114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:51.907413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:51.909588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:51.948246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 845/1687] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:51 2022] POST /identity/v3/projects => generated 407 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:51.949332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:51.952747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:51.969799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd331a4-4ae3-40a1-9d9a-c341b4373389 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:51.980220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.027541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4ad41-e8f2-4723-89a9-7e72225386ca admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.031575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.032197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.032354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.032491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.032725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.032994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.033266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.033536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] PATH_INFO: `/v3/projects/dcd5d2e356d34467912289d176b4923c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.034661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd5d2e356d34467912289d176b4923c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.036698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.045221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.046720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.056618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 843/1688] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:52.057376 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.072199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95316-f222-4e7b-9739-2c7b99ffd397 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.083203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.083203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.083513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.083780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.084427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.086681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.103443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.110417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd5d2e356d34467912289d176b4923c', 'request_id': 'req-093e638a-4e9a-4950-9ee3-686cf68cb52f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.114654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 844/1689] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:52 2022] POST /identity/v3/users => generated 310 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:52.119090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.128449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd5d2e356d34467912289d176b4923c', 'request_id': 'req-093e638a-4e9a-4950-9ee3-686cf68cb52f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.134158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba5396-2f8c-4e17-a141-e40fe81362cb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.143174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd5d2e356d34467912289d176b4923c', 'request_id': 'req-093e638a-4e9a-4950-9ee3-686cf68cb52f'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.166031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] The token cache is being invalidate because project dcd5d2e356d34467912289d176b4923c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:52.166310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.167192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.169211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 846/1690] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:51 2022] DELETE /identity/v3/projects/dcd5d2e356d34467912289d176b4923c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.173086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.175222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.230159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.230824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.231084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.231384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.232046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.233635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.244954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 845/1691] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:52.249393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.251113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.264113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57f8f-67fd-4ce4-9dd0-3d7dd466f962 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.269007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.269678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.269953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.270224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] PATH_INFO: `/v3/projects/3bfbd75704b44c7894471057856af9ee/users/c62a11e5554b487c8cfcff741632292a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.271066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e638a-4e9a-4950-9ee3-686cf68cb52f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.276172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.277022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.277126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.277393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] PATH_INFO: `/v3/users/ae019b6073a3432fa203b78300a6aa6a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.278590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae019b6073a3432fa203b78300a6aa6a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.280673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.287949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfbd75704b44c7894471057856af9ee, user_id=c62a11e5554b487c8cfcff741632292a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.290619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.307904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 846/1692] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:52 2022] PUT /identity/v3/projects/3bfbd75704b44c7894471057856af9ee/users/c62a11e5554b487c8cfcff741632292a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.309052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.340282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae019b6073a3432fa203b78300a6aa6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.355908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae019b6073a3432fa203b78300a6aa6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.363674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae019b6073a3432fa203b78300a6aa6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.364387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae019b6073a3432fa203b78300a6aa6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.364998 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae019b6073a3432fa203b78300a6aa6a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.388632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e8ddf-b96c-4b4c-a66c-473424981ccb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.389919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 847/1693] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:52 2022] DELETE /identity/v3/users/ae019b6073a3432fa203b78300a6aa6a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.390902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.393505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.407986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.462774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.474721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.475617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.476037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.476328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.477035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.478783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.480080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5626e2-e6cc-4587-8475-ebee200ab61b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.486437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.486866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.487265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.487495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] PATH_INFO: `/v3/projects/02f70c4d586949678f615292c7d49957` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.488586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f70c4d586949678f615292c7d49957)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.490786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.490927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 847/1694] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:52.495055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.497089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.509520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c543-4f81-46fd-b19e-fac54824f1e0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.514461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.515380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.515663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.516043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] PATH_INFO: `/v3/projects/3bfbd75704b44c7894471057856af9ee/users/c62a11e5554b487c8cfcff741632292a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.517844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfbd75704b44c7894471057856af9ee, user_id=c62a11e5554b487c8cfcff741632292a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.520585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f9d0f-f2c0-419e-bc82-91103bfde7ed admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.542922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 848/1695] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:52 2022] PUT /identity/v3/projects/3bfbd75704b44c7894471057856af9ee/users/c62a11e5554b487c8cfcff741632292a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.544465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.544756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.545026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.545713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f70c4d586949678f615292c7d49957', 'request_id': 'req-eb47d742-e642-4de9-8357-a1ed5173dd83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.552648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f70c4d586949678f615292c7d49957', 'request_id': 'req-eb47d742-e642-4de9-8357-a1ed5173dd83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.568324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f70c4d586949678f615292c7d49957', 'request_id': 'req-eb47d742-e642-4de9-8357-a1ed5173dd83'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.582800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] MFA Rules not processed for user `c62a11e5554b487c8cfcff741632292a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:52.583564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] The token cache is being invalidate because project 02f70c4d586949678f615292c7d49957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:52.584011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.584122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.584698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.586568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 848/1696] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:52 2022] DELETE /identity/v3/projects/02f70c4d586949678f615292c7d49957 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.590714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.591972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.659203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 849/1697] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:52 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:52.669487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47d742-e642-4de9-8357-a1ed5173dd83 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.670145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.671687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.674726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.675405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.675687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.675961 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] PATH_INFO: `/v3/users/7a4afb5d615c4679a41f04ae67e974de` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.677336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4afb5d615c4679a41f04ae67e974de)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.680353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.744248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd9ed2-3309-48f6-854b-55bf87023bb7 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.748455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.749075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.749352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.749691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.752824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4afb5d615c4679a41f04ae67e974de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.760360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.762144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.766380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4afb5d615c4679a41f04ae67e974de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.774169 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4afb5d615c4679a41f04ae67e974de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.774777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 850/1698] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:52.775888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.777366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.784755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4afb5d615c4679a41f04ae67e974de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.786361 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-356bfe71-e556-4579-832b-788263411ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4afb5d615c4679a41f04ae67e974de'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.793103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e5611-9e4e-487d-abfa-186661b0740e service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:52.799372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:52.800670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.801168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.801522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] PATH_INFO: `/v3/projects/2a818a879cd34f93967de4f6b7527cd3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.803141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a818a879cd34f93967de4f6b7527cd3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:52.805328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:52.808041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 849/1699] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:52 2022] DELETE /identity/v3/users/7a4afb5d615c4679a41f04ae67e974de => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:52.839843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.872932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a818a879cd34f93967de4f6b7527cd3', 'request_id': 'req-cf7bf3fd-900c-4344-b195-628450385ef8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.882338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a818a879cd34f93967de4f6b7527cd3', 'request_id': 'req-cf7bf3fd-900c-4344-b195-628450385ef8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.895635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a818a879cd34f93967de4f6b7527cd3', 'request_id': 'req-cf7bf3fd-900c-4344-b195-628450385ef8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.919233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] The token cache is being invalidate because project 2a818a879cd34f93967de4f6b7527cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:52.919704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.920585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:52.922719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 851/1700] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:52 2022] DELETE /identity/v3/projects/2a818a879cd34f93967de4f6b7527cd3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:52.928263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a04-0249-410f-b688-9aa1aeb3e458 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:52.929256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a04-0249-410f-b688-9aa1aeb3e458 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:52.929534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a04-0249-410f-b688-9aa1aeb3e458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:52.938552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:52.952879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bac65a04-0249-410f-b688-9aa1aeb3e458 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:52.965159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bac65a04-0249-410f-b688-9aa1aeb3e458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:52.968796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:52.970310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.052607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 850/1701] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:53.060936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.061562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.062244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.086545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:53.087859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.088080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bf3fd-900c-4344-b195-628450385ef8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.092753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.093547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.093875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.094151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.095986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.098547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 851/1702] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:53.102423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.116686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5675b97-f0b7-4655-bece-9a76aa2e851d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.124747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.125533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.125877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.126369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.127138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:53.127595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.130666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.148391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 852/1703] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:53.163608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.178956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.179855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979edec-431c-40c7-945f-8217b11a6cf6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.180579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.185299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.186040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.186351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.186553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.187230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.189271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.193537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 852/1704] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:53.194423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.196033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.211715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fdd13-4776-4cbd-83a8-d7a9d78546c6 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.218073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.218962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.219522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.220042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] PATH_INFO: `/v3/projects/ee02ecb430ba403da81000322dedfdf6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.221591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee02ecb430ba403da81000322dedfdf6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.224759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.234230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 853/1705] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:53 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:53.238114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.332663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35304d-88d5-4b68-91ee-4ed3ebaae2b1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.337033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee02ecb430ba403da81000322dedfdf6', 'request_id': 'req-5c989cf2-23ba-44b9-a714-23bb58191959'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.349714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee02ecb430ba403da81000322dedfdf6', 'request_id': 'req-5c989cf2-23ba-44b9-a714-23bb58191959'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.356662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:53.366824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee02ecb430ba403da81000322dedfdf6', 'request_id': 'req-5c989cf2-23ba-44b9-a714-23bb58191959'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.386515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] The token cache is being invalidate because project ee02ecb430ba403da81000322dedfdf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:53.386912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.387500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.394288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 853/1706] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:53 2022] DELETE /identity/v3/projects/ee02ecb430ba403da81000322dedfdf6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:53.399010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.400991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.416124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.416560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.416839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.417108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.417748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.420254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.475706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:53.493071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 854/1707] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:53 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:53.494071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.496291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.514160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765dd94-a0eb-4651-bad9-5e26ade65682 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.523677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.524378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.524652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.524918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.525611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.527289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.540935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 855/1708] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:53.543912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c989cf2-23ba-44b9-a714-23bb58191959 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.545382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.550409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.551447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.551656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.552425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] PATH_INFO: `/v3/users/ff466b8e7b1b4541b23b40a7ed968080` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.553644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff466b8e7b1b4541b23b40a7ed968080)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.556304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.559492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9bbe-d757-4fa2-b16d-f211ca845b50 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.566277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.566688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.567048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.567382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.568115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.570653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.589679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 856/1709] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:53.590967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.611105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7715507-808e-4cde-93e4-474dc4583f59 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:53.628275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff466b8e7b1b4541b23b40a7ed968080'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.631781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:53.632502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.632784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.633055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] PATH_INFO: `/v3/projects/7f1e892cb9b643f8ba6f9de5ea6837e6/users/1db4631787b949438e92dc0e34fdb8f3/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.650713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff466b8e7b1b4541b23b40a7ed968080'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.651466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff466b8e7b1b4541b23b40a7ed968080'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.657280 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1e892cb9b643f8ba6f9de5ea6837e6, user_id=1db4631787b949438e92dc0e34fdb8f3, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:53.661657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:53.662099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff466b8e7b1b4541b23b40a7ed968080'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.662804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-16782466-d81f-462a-8bd0-64a34990da43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff466b8e7b1b4541b23b40a7ed968080'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:53.682939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 854/1710] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:24:53 2022] DELETE /identity/v3/users/ff466b8e7b1b4541b23b40a7ed968080 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:53.692667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 857/1711] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:53 2022] PUT /identity/v3/projects/7f1e892cb9b643f8ba6f9de5ea6837e6/users/1db4631787b949438e92dc0e34fdb8f3/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:53.695436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779e36-7946-43be-9252-05e2435819eb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:53.695709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779e36-7946-43be-9252-05e2435819eb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:53.695984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779e36-7946-43be-9252-05e2435819eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:53.732869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48779e36-7946-43be-9252-05e2435819eb None None] MFA Rules not processed for user `1db4631787b949438e92dc0e34fdb8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:53.734163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48779e36-7946-43be-9252-05e2435819eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.811350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:53.823594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 855/1712] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:53.828756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:53.830775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:53.939721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0025312-c110-41d1-853b-556343dfc88e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.032406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779e36-7946-43be-9252-05e2435819eb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.033858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48779e36-7946-43be-9252-05e2435819eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.059721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.060363 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.060645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.061093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.061624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.065660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade7819-9480-40d9-8e7d-1c515712d8e0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.104281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 858/1713] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:53 2022] POST /identity/v3/projects => generated 377 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:54.106081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6c3b-09cb-43a2-b49c-9a622f6ce8f2 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.106431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6c3b-09cb-43a2-b49c-9a622f6ce8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.106879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6c3b-09cb-43a2-b49c-9a622f6ce8f2 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.108454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 859/1714] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:24:54.109378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd6c3b-09cb-43a2-b49c-9a622f6ce8f2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.162677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779e36-7946-43be-9252-05e2435819eb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.176387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:54.209843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd6c3b-09cb-43a2-b49c-9a622f6ce8f2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.216299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.217420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.217420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.217658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.218404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.220695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37ef4d-6fbc-485f-a07d-56e8e6751fec admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.248374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.250698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.250835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 860/1715] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:54.251064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.251387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] PATH_INFO: `/v3/projects/05b1030d830e48818710063e024f7e49` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.252206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.252496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.252829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.256011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b1030d830e48818710063e024f7e49)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.258325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.277923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:54.279376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.349518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b1030d830e48818710063e024f7e49', 'request_id': 'req-397dd253-2476-4cdd-8441-34b967d8d60d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:54.358742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b1030d830e48818710063e024f7e49', 'request_id': 'req-397dd253-2476-4cdd-8441-34b967d8d60d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:54.369265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b1030d830e48818710063e024f7e49', 'request_id': 'req-397dd253-2476-4cdd-8441-34b967d8d60d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:54.382501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 861/1716] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:54.383272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] The token cache is being invalidate because project 05b1030d830e48818710063e024f7e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:24:54.383699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.383979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:54.385271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-397dd253-2476-4cdd-8441-34b967d8d60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:24:54.387195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 856/1717] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:24:54 2022] DELETE /identity/v3/projects/05b1030d830e48818710063e024f7e49 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:54.394177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.394516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.394984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.410189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e947-ab69-4077-8261-79dc510d7966 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.438926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:54.440688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.505234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:54.512596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.513291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.513720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.514050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.514769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.516784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.529145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 862/1718] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:54.533312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.535063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.584278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 857/1719] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:54.585294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.587680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.607843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3f8c8-c038-46cd-9322-1a8a89889c12 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.615468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.616368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.617494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.617494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.618259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.620957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f5ecc-2de3-45cc-9733-ad07f87c7dd8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.626367 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:54.634691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 858/1720] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:54.636305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.636802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.637070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.651441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98d2d-94b6-43f9-b31a-7a55e28c7930 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.657897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.658533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.658817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.659091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.661424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.661549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:54.663329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.671865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 859/1721] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:54.675129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.683123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c7ca3-10c2-47a7-a988-c7d7cfbb2d2f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.688933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.689599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.690717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.691308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.691577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.691962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] PATH_INFO: `/v3/projects/0ffb72ce2a964ae4b8321489c07b11a7/users/f764399f8f3d4e549dd7cfae30714317/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.707497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 863/1722] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:24:54.708541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ffb72ce2a964ae4b8321489c07b11a7, user_id=f764399f8f3d4e549dd7cfae30714317, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.708857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.711351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724f19-12ab-44d7-8d7a-2a47b903bcb2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.723481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc27e50-aa88-458b-9fe0-74cd18f0846d service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.741919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 860/1723] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:54 2022] PUT /identity/v3/projects/0ffb72ce2a964ae4b8321489c07b11a7/users/f764399f8f3d4e549dd7cfae30714317/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:54.743969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.744571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.745023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.783679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] MFA Rules not processed for user `f764399f8f3d4e549dd7cfae30714317`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:54.785825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:54.818825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.819624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.819970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.820493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.821221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:24:54.821692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.824013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.842523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 864/1724] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:24:54.844049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.863604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bb9b5-2df1-4882-bb6f-85bfdc96ba70 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.870294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:54.871258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:54.871544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:54.871818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:54.872471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:54.875423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:54.888158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 861/1725] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:54.893010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:54.917723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50193dd7-c65a-4666-8630-cd8cb7d64510 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:54.920563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 865/1726] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/projects => generated 393 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:54.921939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.013030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093d3a6-6a15-4089-ae5f-8c4209c40e4d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.021261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.021946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.022453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.022554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.027544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.030524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.077141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.077924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.078350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.078614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.079268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.081505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.090753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 866/1727] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:55.095096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.109575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392782de-6290-4f30-9c3a-5ea69747efa8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.115713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.116595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.117075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.117631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.118642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.120741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.140095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 867/1728] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:55.140245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 862/1729] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:54 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:55.142558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.143456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.162116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:55.170142 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553b19-4f7f-4c96-b3e9-84ab7389eaf7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.231538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:55.272872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.273917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.274248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.274519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.275301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.277423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.284448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f198e48-e51c-4acc-98b7-55831848dbfb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.289354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 868/1730] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:55.293603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.299073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.300703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.301130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.301426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.303992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.306024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.307416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df48e0-e7f0-48a9-924c-393843051fbe admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.313808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.314579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.314911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.315208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] PATH_INFO: `/v3/projects/4d94cbd1b4f04055b5a632b45398de25/users/4180e4de8e154097b4b7e9f1e114eaa9/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.339906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d94cbd1b4f04055b5a632b45398de25, user_id=4180e4de8e154097b4b7e9f1e114eaa9, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.344681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d23628-c2f1-4bf3-b564-11c4df3ef297 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.357786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 863/1731] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:55 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:55.359315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.368344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 869/1732] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:55 2022] PUT /identity/v3/projects/4d94cbd1b4f04055b5a632b45398de25/users/4180e4de8e154097b4b7e9f1e114eaa9/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:55.381442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.382560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.382974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.393141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418ae63-15b8-4235-a533-cc1995a4d4a3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.416675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] MFA Rules not processed for user `4180e4de8e154097b4b7e9f1e114eaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:55.417717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:55.469004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.469634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.469896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.470165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.470816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.472636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.484885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 864/1733] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:24:55.490387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.500035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 870/1734] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:55.504798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.509699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07768-62da-42d7-bbf5-56919cbcffce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.515549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.516435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.516721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.516912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aade4b7-55a8-4054-9956-e4ca0b50d397 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.517132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] PATH_INFO: `/v3/projects/ff751a524f4748939d77fc381b0026da/users/bc32157d7f49433fa15efeafddbc4d7b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.530494 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.531347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.531715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.532192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.532919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.535155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.546132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff751a524f4748939d77fc381b0026da, user_id=bc32157d7f49433fa15efeafddbc4d7b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.551861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.573180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 871/1735] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:24:55 2022] POST /identity/v3/projects => generated 393 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:24:55.577703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.587684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 865/1736] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:55 2022] PUT /identity/v3/projects/ff751a524f4748939d77fc381b0026da/users/bc32157d7f49433fa15efeafddbc4d7b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:24:55.588918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.680097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cef7-19de-4b57-8b05-e0a12b72e0f9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.688300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215e0fe-90e0-4c5e-bfaa-59bf7c51914d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.688421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.688978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.689343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.689517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.690156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.692112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.693100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.693863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.694156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.694415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.695066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.696740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.712248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 866/1737] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:55.714303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.725731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:55.730548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921b954-a111-45e6-9f1d-2c8539986a90 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.735492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 872/1738] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:24:55 2022] POST /identity/v3/users => generated 303 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:24:55.736716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.737081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.737806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.738262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.738861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] PATH_INFO: `/v3/projects/ff751a524f4748939d77fc381b0026da/users/bc32157d7f49433fa15efeafddbc4d7b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.741836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff751a524f4748939d77fc381b0026da, user_id=bc32157d7f49433fa15efeafddbc4d7b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.745943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc68cd2-783f-4f3e-9fce-93ab94bfc047 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.754433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cbecb-2a4a-4160-a430-8300611ff57b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.759457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 867/1739] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:55 2022] PUT /identity/v3/projects/ff751a524f4748939d77fc381b0026da/users/bc32157d7f49433fa15efeafddbc4d7b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:55.770579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:55.772745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.773115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.773305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.809965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] MFA Rules not processed for user `bc32157d7f49433fa15efeafddbc4d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:55.811284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:55.862711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.864195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.864688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.865090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.866130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.869548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.884207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 873/1740] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:55.885328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.918173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bbaa8-768a-4df5-ac6f-d919007c816b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:55.928165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:55.928866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:55.929309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:55.929467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] PATH_INFO: `/v3/projects/6f3fb8e7d78642c9b73b533e4483f319/users/396c3c65e4534cb991504bc4fe1b9209/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:55.931586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 868/1741] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:24:55.945159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:55.946329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3fb8e7d78642c9b73b533e4483f319, user_id=396c3c65e4534cb991504bc4fe1b9209, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:55.946806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:55.949111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:55.963008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 874/1742] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:55 2022] PUT /identity/v3/projects/6f3fb8e7d78642c9b73b533e4483f319/users/396c3c65e4534cb991504bc4fe1b9209/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:55.966801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:56.005493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e256-b49a-4797-8584-8b93809d9466 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.027972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3aad28-a391-42f6-bf90-deec3915874d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.032729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.033409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.033707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.034046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.034698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.036337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.046324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 875/1743] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:24:56.050434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:56.054994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.055652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.055927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.056198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.060087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a47df5-40af-4a73-9ee2-621f8bb38ee6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.064770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.065426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.065701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.065942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] PATH_INFO: `/v3/projects/6f3fb8e7d78642c9b73b533e4483f319/users/396c3c65e4534cb991504bc4fe1b9209/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.067892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f3fb8e7d78642c9b73b533e4483f319, user_id=396c3c65e4534cb991504bc4fe1b9209, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.070514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff77efd-897d-474e-9a72-246fc94e8915 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.083496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 876/1744] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:24:56 2022] PUT /identity/v3/projects/6f3fb8e7d78642c9b73b533e4483f319/users/396c3c65e4534cb991504bc4fe1b9209/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:24:56.093316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.093608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.093889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.110153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:56.134914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] MFA Rules not processed for user `396c3c65e4534cb991504bc4fe1b9209`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:24:56.136461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:56.136916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.138382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.153105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 869/1745] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:56.187939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:24:56.197102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:56.198653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:56.209953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a48040-fe73-4f2b-9832-86f558930258 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.210616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 877/1746] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:24:56 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:24:56.222049 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:56.264811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.265441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.265706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.265976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.267803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:24:56.278126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f2612-2dac-4a1a-8d22-5776bbbe8ebc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.281956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.282532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.282810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.283078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.292802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.294256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.303911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 878/1747] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:56.329722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.331188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.346018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 870/1748] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:24:56.354523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:24:56.365326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601474f4-76c5-4486-a457-a8ba0e21489d service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:24:56.369639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:24:56.370271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:24:56.370680 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:24:56.370769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:24:56.433582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:24:56.435244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb7911-34f1-4810-bf94-a3912d7a8862 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:24:56.446923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 879/1749] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:24:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:04.190391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:04.191949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:04.266208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5adde-4183-41be-ab6b-b4659ac33970 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:04.270856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:04.271541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:04.272107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:04.272270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] PATH_INFO: `/v3/users/8b15ee6e86e44c059320b2f765683ef7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:04.273375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b15ee6e86e44c059320b2f765683ef7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:04.275349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:04.319393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:04.327994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b15ee6e86e44c059320b2f765683ef7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.341478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b15ee6e86e44c059320b2f765683ef7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.342224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b15ee6e86e44c059320b2f765683ef7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.350208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b15ee6e86e44c059320b2f765683ef7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.350835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b15ee6e86e44c059320b2f765683ef7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.363909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 871/1750] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:04 2022] DELETE /identity/v3/users/8b15ee6e86e44c059320b2f765683ef7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:04.368172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:04.368611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:04.368716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:04.388188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:04.389307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:04.451272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 880/1751] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:04.461811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:04.463279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:04.521150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba47b-b86a-4116-9c4b-d6cb44320768 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:04.524925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:04.525547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:04.525830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:04.526099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:04.535484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:04.537169 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:04.546658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 872/1752] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:04.875891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:04.877674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:04.897402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d43c0-18ac-4667-9ba5-444c45712521 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:04.902098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:04.902812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:04.903183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:04.903562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] PATH_INFO: `/v3/projects/f7c1554d89c54eef84ff326e6b2b198a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:04.904746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c1554d89c54eef84ff326e6b2b198a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:04.906730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:04.932489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:04.956756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c1554d89c54eef84ff326e6b2b198a', 'request_id': 'req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.963321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c1554d89c54eef84ff326e6b2b198a', 'request_id': 'req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.971470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c1554d89c54eef84ff326e6b2b198a', 'request_id': 'req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.982121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] The token cache is being invalidate because project f7c1554d89c54eef84ff326e6b2b198a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:04.982405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.983017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:04.984575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 881/1753] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:04 2022] DELETE /identity/v3/projects/f7c1554d89c54eef84ff326e6b2b198a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:04.988706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:04.998510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e592-6918-4416-b0bc-360820697320 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.056554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.057194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.057479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.057753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] PATH_INFO: `/v3/users/c62a11e5554b487c8cfcff741632292a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.059067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62a11e5554b487c8cfcff741632292a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.061100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.111865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62a11e5554b487c8cfcff741632292a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.123660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:05.124483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62a11e5554b487c8cfcff741632292a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.125233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62a11e5554b487c8cfcff741632292a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.133680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62a11e5554b487c8cfcff741632292a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.134566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9eff8cb2-84bc-4253-b5e2-bdaac20303ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62a11e5554b487c8cfcff741632292a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.150355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 873/1754] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:04 2022] DELETE /identity/v3/users/c62a11e5554b487c8cfcff741632292a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:05.478881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:05.480377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:05.496215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a652d-8e2c-4f85-9cbb-f6040ab4a53a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.555877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.556523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.556804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.557076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] PATH_INFO: `/v3/projects/3bfbd75704b44c7894471057856af9ee` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.558231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bfbd75704b44c7894471057856af9ee)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.560199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.608563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bfbd75704b44c7894471057856af9ee', 'request_id': 'req-738afc37-a691-4681-b3c6-13f3c63075ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.614831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bfbd75704b44c7894471057856af9ee', 'request_id': 'req-738afc37-a691-4681-b3c6-13f3c63075ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.624387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bfbd75704b44c7894471057856af9ee', 'request_id': 'req-738afc37-a691-4681-b3c6-13f3c63075ba'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.635915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] The token cache is being invalidate because project 3bfbd75704b44c7894471057856af9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:05.636256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.636905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-738afc37-a691-4681-b3c6-13f3c63075ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:05.638570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 882/1755] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:05 2022] DELETE /identity/v3/projects/3bfbd75704b44c7894471057856af9ee => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:05.643620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.644046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.644170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.663857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:05.664977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:05.727040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 874/1756] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:05.735605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.735893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.736166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.758138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:05.759415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:05.768911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 883/1757] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:05.773550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:05.784048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256af0e-5df1-4fdd-9599-4772307bfb0d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.790778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.791426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.791706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.791969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.792615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:05.792917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.794610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.811546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 875/1758] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:05.813900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:05.823930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c07327-678b-4d10-8881-f2a839858fab None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.828831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.829454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.829723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.829996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.830628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.833030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.861794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 884/1759] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:05.866198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:05.878974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30114519-8e49-4694-a468-0c3c46f6bc99 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.938296 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.938912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.939223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.939492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.940128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.942035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.964760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 876/1760] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:05 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:05.968732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:05.978963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35403b9b-8b51-4de6-a371-e5c9d2d92b60 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:05.983547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:05.984189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:05.984443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:05.984736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:05.985381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:05.986993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:05.998122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 885/1761] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:06.002173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.012372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682cb0c-6f44-4347-92a7-724323f68541 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.017295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.018008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.018352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.018695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.019462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.021114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb822-83d3-46cf-9df0-81e1834c399f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.033208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 877/1762] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:06.037648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.042998 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:06.050300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b2a7c-e04d-4121-ab31-eeb0bab31d10 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.056706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.057537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.057922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.058409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] PATH_INFO: `/v3/projects/a748e37ed7bc4fe3a91fc69b281236fa/users/9dc5e8a8fc984a22bf3ad3bf743ba2db/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.074167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a748e37ed7bc4fe3a91fc69b281236fa, user_id=9dc5e8a8fc984a22bf3ad3bf743ba2db, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.076912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.092987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 886/1763] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:06 2022] PUT /identity/v3/projects/a748e37ed7bc4fe3a91fc69b281236fa/users/9dc5e8a8fc984a22bf3ad3bf743ba2db/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:06.103465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.103745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.104009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.133246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] MFA Rules not processed for user `9dc5e8a8fc984a22bf3ad3bf743ba2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:06.134466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:06.168888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:06.203363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 878/1764] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:06.208334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.274682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb6a2-3308-4747-b938-70ff45c988ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.280091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.280703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.280985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.281252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.281885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.283849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.316249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 887/1765] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:06 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:06.317516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.328781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24289b2c-145f-4378-9677-3adca96bbe3c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.395747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.396508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.396788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.397057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.397700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.399657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.425311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 879/1766] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:06 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:06.426536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.437288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71d33a-f9e9-4a3d-8dfd-d270cb0fa942 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.444719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.445331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.445610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.445881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.446535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.448174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.460540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 888/1767] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:06.464878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.475209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15723cf-bc5d-443a-ba57-e75d7ca23875 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.480230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.480895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.481171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.481445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] PATH_INFO: `/v3/projects/834e0a3ea43042fb9c98d9503ffa3533/users/0d7a74b9c5304afdb48227a23244ddac/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.496194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834e0a3ea43042fb9c98d9503ffa3533, user_id=0d7a74b9c5304afdb48227a23244ddac, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.498936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.513078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 880/1768] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:06 2022] PUT /identity/v3/projects/834e0a3ea43042fb9c98d9503ffa3533/users/0d7a74b9c5304afdb48227a23244ddac/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:06.517190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.527739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1172859-fc1a-4863-b919-edc24d3b5032 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.583933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.584541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.584820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.585091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.585738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.587368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589d176-2149-409c-be47-977ae8be2d0c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.597052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 889/1769] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:06.601480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.612838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cf484-b344-41e0-8005-9e51f36c60a1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.618085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.618756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.619039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.619348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] PATH_INFO: `/v3/projects/834e0a3ea43042fb9c98d9503ffa3533/users/0d7a74b9c5304afdb48227a23244ddac/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.621421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834e0a3ea43042fb9c98d9503ffa3533, user_id=0d7a74b9c5304afdb48227a23244ddac, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.624326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.635690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 881/1770] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:06 2022] PUT /identity/v3/projects/834e0a3ea43042fb9c98d9503ffa3533/users/0d7a74b9c5304afdb48227a23244ddac/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:06.647277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.647568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.647844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.676662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] MFA Rules not processed for user `0d7a74b9c5304afdb48227a23244ddac`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:06.677809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:06.716561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:06.749531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 890/1771] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:06.760292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:06.762158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:06.818426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4837c2-b67c-4518-98e9-b27ea816aeb0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:06.822759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:06.823397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:06.823673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:06.823947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:06.867359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:06.898034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:06.899509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78408dfe-6e49-4466-b49a-54e589856ca3 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:06.908443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 882/1772] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:08.662765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:08.664277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:08.738513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74c991-e7ec-41d6-9c74-f56897a4b9f9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:08.743940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:08.744583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:08.744869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:08.745132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] PATH_INFO: `/v3/users/4180e4de8e154097b4b7e9f1e114eaa9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:08.746277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4180e4de8e154097b4b7e9f1e114eaa9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:08.748254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:08.799667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4180e4de8e154097b4b7e9f1e114eaa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:08.811927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4180e4de8e154097b4b7e9f1e114eaa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:08.819074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4180e4de8e154097b4b7e9f1e114eaa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:08.819825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4180e4de8e154097b4b7e9f1e114eaa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:08.820421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4180e4de8e154097b4b7e9f1e114eaa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:08.834412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 891/1773] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:08 2022] DELETE /identity/v3/users/4180e4de8e154097b4b7e9f1e114eaa9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:08.839233 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:08.839742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:08.840017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:08.861600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:08.862959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:08.929615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 883/1774] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:08.942826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:08.944286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.037530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404f53-1e78-4a3a-a80e-54d70674aaf3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:09.042091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:09.042746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.043025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.043366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.053616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:09.064883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:09.071228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210091b-3ecf-49b8-913d-06d0aeaf8b71 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:09.089415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 892/1775] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:09.097475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:09.100344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.113850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf0118-c945-4ffd-923b-0ebf58389908 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:09.122195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:09.123820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.124231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.124514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] PATH_INFO: `/v3/users/1db4631787b949438e92dc0e34fdb8f3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.126437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db4631787b949438e92dc0e34fdb8f3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:09.128620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:09.207684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db4631787b949438e92dc0e34fdb8f3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.224676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db4631787b949438e92dc0e34fdb8f3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.226444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db4631787b949438e92dc0e34fdb8f3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.237090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db4631787b949438e92dc0e34fdb8f3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.237981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db4631787b949438e92dc0e34fdb8f3'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.254145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 884/1776] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:09 2022] DELETE /identity/v3/users/1db4631787b949438e92dc0e34fdb8f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:09.258657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.258949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.259272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.316763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:09.318656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.396917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 893/1777] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:09.408181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:09.409863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.432582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:09.434295 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.447637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c18ef-f6d4-475d-9b5b-251854ae2a24 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:09.453220 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:09.453873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.454193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.454410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] PATH_INFO: `/v3/projects/4d94cbd1b4f04055b5a632b45398de25` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.455761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d94cbd1b4f04055b5a632b45398de25)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:09.457860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:09.511895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d94cbd1b4f04055b5a632b45398de25', 'request_id': 'req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.520010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d94cbd1b4f04055b5a632b45398de25', 'request_id': 'req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.527055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aeb17-da29-4d64-a25e-084c3100e2b3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:09.531923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d94cbd1b4f04055b5a632b45398de25', 'request_id': 'req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.535101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:09.548261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] The token cache is being invalidate because project 4d94cbd1b4f04055b5a632b45398de25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:09.548547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.549458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.552250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 894/1778] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:09 2022] DELETE /identity/v3/projects/4d94cbd1b4f04055b5a632b45398de25 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:09.556320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:09.558015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.595056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:09.595714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.596134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.596319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.598102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:09.649483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb8630-ca10-41a5-87c7-ef4d9de4b2b9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:09.654408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:09.655321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:09.655597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:09.655869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] PATH_INFO: `/v3/users/396c3c65e4534cb991504bc4fe1b9209` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:09.657002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396c3c65e4534cb991504bc4fe1b9209)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:09.658944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:09.669886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:09.671443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:09.681754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 885/1779] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:09.684816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:09.719712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396c3c65e4534cb991504bc4fe1b9209'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.733325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396c3c65e4534cb991504bc4fe1b9209'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.741227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396c3c65e4534cb991504bc4fe1b9209'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.742019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396c3c65e4534cb991504bc4fe1b9209'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.742640 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396c3c65e4534cb991504bc4fe1b9209'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:09.757368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 895/1780] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:09 2022] DELETE /identity/v3/users/396c3c65e4534cb991504bc4fe1b9209 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:10.171845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.171845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.173686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ad773-d3a4-48b8-a749-85e6c6956695 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.238090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.239839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.273573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.274367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.274722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.274902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] PATH_INFO: `/v3/projects/7f1e892cb9b643f8ba6f9de5ea6837e6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.276183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1e892cb9b643f8ba6f9de5ea6837e6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.278293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.318031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921f966-3802-49e1-b674-e0f7e182fd45 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.329569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.330558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.330837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.331764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] PATH_INFO: `/v3/projects/6f3fb8e7d78642c9b73b533e4483f319` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.333463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f3fb8e7d78642c9b73b533e4483f319)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.335534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.351689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1e892cb9b643f8ba6f9de5ea6837e6', 'request_id': 'req-99693dc6-ff48-45fa-86dc-4dafe848809e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.361802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1e892cb9b643f8ba6f9de5ea6837e6', 'request_id': 'req-99693dc6-ff48-45fa-86dc-4dafe848809e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.371834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1e892cb9b643f8ba6f9de5ea6837e6', 'request_id': 'req-99693dc6-ff48-45fa-86dc-4dafe848809e'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.373335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:10.387123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] The token cache is being invalidate because project 7f1e892cb9b643f8ba6f9de5ea6837e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:10.387639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.388627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.390219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 886/1781] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:10 2022] DELETE /identity/v3/projects/7f1e892cb9b643f8ba6f9de5ea6837e6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:10.394009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.394237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f3fb8e7d78642c9b73b533e4483f319', 'request_id': 'req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.395518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.401099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f3fb8e7d78642c9b73b533e4483f319', 'request_id': 'req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.409505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f3fb8e7d78642c9b73b533e4483f319', 'request_id': 'req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.420687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] The token cache is being invalidate because project 6f3fb8e7d78642c9b73b533e4483f319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:10.421016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.421631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-03b3b612-52f2-4ee7-bd9e-d1b458b63ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.423138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 896/1782] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:10 2022] DELETE /identity/v3/projects/6f3fb8e7d78642c9b73b533e4483f319 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:10.428341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7441-7938-4e53-9848-f47ca3c95c61 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.428647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7441-7938-4e53-9848-f47ca3c95c61 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.429014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7441-7938-4e53-9848-f47ca3c95c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.449124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-866c7441-7938-4e53-9848-f47ca3c95c61 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:10.450325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-866c7441-7938-4e53-9848-f47ca3c95c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.461568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99693dc6-ff48-45fa-86dc-4dafe848809e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.477229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:10.514323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 897/1783] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:10.516800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.517472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.517749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.518016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] PATH_INFO: `/v3/users/f764399f8f3d4e549dd7cfae30714317` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.519174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f764399f8f3d4e549dd7cfae30714317)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.521118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.521693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.521974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.522241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.542518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:10.543887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.554198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 898/1784] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:10.558854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.568375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab201a-6791-4411-bebd-3b975ade9dda None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.571850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f764399f8f3d4e549dd7cfae30714317'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.573439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.574404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.574685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.574961 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.575648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:10.576083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.577726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.585077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f764399f8f3d4e549dd7cfae30714317'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.585786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f764399f8f3d4e549dd7cfae30714317'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.590310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 899/1785] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:25:10.594949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.597425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.597723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f764399f8f3d4e549dd7cfae30714317'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.598834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f764399f8f3d4e549dd7cfae30714317'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:10.612593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34376-0cc0-473a-8656-7641a3d5b6e7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.616327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 887/1786] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:10 2022] DELETE /identity/v3/users/f764399f8f3d4e549dd7cfae30714317 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:10.675173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.675824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.676101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.676372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.677015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.678968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.710420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 900/1787] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:10 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:10.711701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.721917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b8055-aef8-4319-a04c-1987e8f2f442 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.733791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:10.791762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.792388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.792670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.792944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.793590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.795570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.818898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 888/1788] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:10 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:10.823016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.836729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a131c-751a-4fea-8d35-d0e6081dfa9a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.859287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.860066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.860290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.860549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.861223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.864254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.878266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 901/1789] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:10.882424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.907825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a3b7c-2711-476c-b3bd-5abdcc38a532 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.919074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.919824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.920155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.920454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.921397 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.929845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.931234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.933197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:10.954076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 889/1790] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:10.970240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccca9c-c41d-4c2d-a374-bca7ed7a9aa3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:10.978010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:10.990640 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:10.991335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:10.991611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:10.991879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] PATH_INFO: `/v3/projects/0ffb72ce2a964ae4b8321489c07b11a7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:10.993194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ffb72ce2a964ae4b8321489c07b11a7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:10.995086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:10.998781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011bc76-2be4-4046-89fa-a2fe77e7234a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.021285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.021980 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.022259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.022532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] PATH_INFO: `/v3/projects/d74aba013735417b907c9bfd7893b3e3/users/15e13d02aaff40e0b1bf83dfd1fcb6df/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.039228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74aba013735417b907c9bfd7893b3e3, user_id=15e13d02aaff40e0b1bf83dfd1fcb6df, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.042382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ddc26-ba29-45d7-8cae-95209314dac3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.063196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 890/1791] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:10 2022] PUT /identity/v3/projects/d74aba013735417b907c9bfd7893b3e3/users/15e13d02aaff40e0b1bf83dfd1fcb6df/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:11.063424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ffb72ce2a964ae4b8321489c07b11a7', 'request_id': 'req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.072026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.072427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.072704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.085603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ffb72ce2a964ae4b8321489c07b11a7', 'request_id': 'req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.099208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ffb72ce2a964ae4b8321489c07b11a7', 'request_id': 'req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.107351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] MFA Rules not processed for user `15e13d02aaff40e0b1bf83dfd1fcb6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:11.108917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.114892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] The token cache is being invalidate because project 0ffb72ce2a964ae4b8321489c07b11a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:11.115707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.116800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.118505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 902/1792] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:10 2022] DELETE /identity/v3/projects/0ffb72ce2a964ae4b8321489c07b11a7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:11.122500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.124099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.135229 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:11.211384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 891/1793] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:11.214268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.215940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.236812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ed119-5e21-4b11-902e-6319a1e3d8e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.326784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2d1a4-701f-4e1a-a2ca-28f6535853b5 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.331700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.332324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.332718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.332945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.333050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.333223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.333322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] PATH_INFO: `/v3/users/bc32157d7f49433fa15efeafddbc4d7b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.333489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.334131 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.334386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc32157d7f49433fa15efeafddbc4d7b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.336100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.336353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.340882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:11.382883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 892/1794] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:11.388178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.403100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc32157d7f49433fa15efeafddbc4d7b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.433548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc32157d7f49433fa15efeafddbc4d7b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.440780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc32157d7f49433fa15efeafddbc4d7b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.441593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc32157d7f49433fa15efeafddbc4d7b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.442201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc32157d7f49433fa15efeafddbc4d7b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.458330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 903/1795] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:11 2022] DELETE /identity/v3/users/bc32157d7f49433fa15efeafddbc4d7b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:11.464499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd741ac3-d515-4f2d-b378-ab476af54b7e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.532876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.533472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.534128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.534128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.534913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.536907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.560661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 893/1796] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/users => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:11.565379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.567071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.587605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45dcb6-69de-45db-8914-130dbccddf6b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.592697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.593647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.593788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.594118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.594859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.596782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.618138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 904/1797] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:11.618349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.631644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c8efc-88aa-4a3a-9e77-2bc1402799ca admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.649077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.650603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.650978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.651386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] PATH_INFO: `/v3/projects/345ce7ea557b458891b71b83db65c75c/users/be9991b4242d4195b9a73fb910a4aa55/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.671597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] RBAC: Authorizing `identity:create_grant(project_id=345ce7ea557b458891b71b83db65c75c, user_id=be9991b4242d4195b9a73fb910a4aa55, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.674517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.695215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 894/1798] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:11 2022] PUT /identity/v3/projects/345ce7ea557b458891b71b83db65c75c/users/be9991b4242d4195b9a73fb910a4aa55/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:11.696688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.707375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26c6f4-bc79-4d0b-9a09-612db4c619f2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.726446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:11.759036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.760940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.771125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.771777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.772055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.772322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.773100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.774709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.784557 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b79bd-5fe3-4391-b230-191d625f407c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.785226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 905/1799] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:11.789277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.793237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.793876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.794154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.794422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] PATH_INFO: `/v3/projects/ff751a524f4748939d77fc381b0026da` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.795577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff751a524f4748939d77fc381b0026da)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.797518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.799606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040e2f-d244-45b7-8afc-89631cae251a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:11.804251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:11.804901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.805178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.805446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] PATH_INFO: `/v3/projects/345ce7ea557b458891b71b83db65c75c/users/be9991b4242d4195b9a73fb910a4aa55/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.807404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] RBAC: Authorizing `identity:create_grant(project_id=345ce7ea557b458891b71b83db65c75c, user_id=be9991b4242d4195b9a73fb910a4aa55, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:11.810079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ce929-6a96-44c7-8ab1-9b9d61ba933a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:11.826144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 906/1800] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:11 2022] PUT /identity/v3/projects/345ce7ea557b458891b71b83db65c75c/users/be9991b4242d4195b9a73fb910a4aa55/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:11.837574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.837859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.838132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.847363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff751a524f4748939d77fc381b0026da', 'request_id': 'req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.849199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:11.855475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff751a524f4748939d77fc381b0026da', 'request_id': 'req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.864955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff751a524f4748939d77fc381b0026da', 'request_id': 'req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.868404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] MFA Rules not processed for user `be9991b4242d4195b9a73fb910a4aa55`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:11.869679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.876485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] The token cache is being invalidate because project ff751a524f4748939d77fc381b0026da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:11.876948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.877693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c604c6eb-a0cb-4d61-b6c9-e235fa3e723d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:11.879402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 895/1801] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:11 2022] DELETE /identity/v3/projects/ff751a524f4748939d77fc381b0026da => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:11.885129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09b2d-4541-4451-822c-00da19a4b82a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.885491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09b2d-4541-4451-822c-00da19a4b82a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.885862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd09b2d-4541-4451-822c-00da19a4b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.905482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3cd09b2d-4541-4451-822c-00da19a4b82a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:11.906671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3cd09b2d-4541-4451-822c-00da19a4b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.939888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 907/1802] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:11.950811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:11.952443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:11.965913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 896/1803] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:11.974433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:11.974706 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:11.975120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:11.994351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:11.995584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.004357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 897/1804] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:12.007903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.017023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abebe93-b975-49ee-8bd8-c9712a944857 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.017204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27a20-e0a7-43b9-9482-3f0a8a2ac11e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.021076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.021703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.021921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.022041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.022244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.022688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.022979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.023319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.023900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:12.024213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.025925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.033218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.035365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.043030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 898/1805] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:25:12.058475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.058769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 908/1806] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:12.059132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f8de-1173-4ba5-b29e-3d93d2f69a6f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.061041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.061896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.062235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.062642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.063300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.065239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.094270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 899/1807] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:12 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:12.099334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.109984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1ee09-3c8f-4738-a5ee-e64fcb0ccb93 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.143308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:12.152949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.172943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.173560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.173834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.174097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.174739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.176687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.200240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 909/1808] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:12 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:12.204408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.213043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b7fe1-862a-43e9-9ccd-39fd1976cab1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.214204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfde58-ea1c-436b-9981-2a7e7c2bb437 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.216993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.217673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.217952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.218219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.219044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.219659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.219936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.220033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.220201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.220843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.222422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.232516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 910/1809] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:12.236443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.245942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78b386-1a55-45ad-bd41-eb63e3eb5183 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.250923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.251532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.251803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.252068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.252723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.254245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.264749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 911/1810] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:12.268962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.278642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ca86-a132-46dd-9498-34e6f474713c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.280095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:12.283958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.284596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.284870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.285028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.285257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] PATH_INFO: `/v3/projects/d4deb529a89341d48d178161cc426865/users/ee899d4497b5423c896fa3d1dfbbda2c/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.286475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7649c0-5014-4450-8a38-cb641821e966 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.296249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 900/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:12.299885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4deb529a89341d48d178161cc426865, user_id=ee899d4497b5423c896fa3d1dfbbda2c, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.302538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.316812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 912/1812] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:12 2022] PUT /identity/v3/projects/d4deb529a89341d48d178161cc426865/users/ee899d4497b5423c896fa3d1dfbbda2c/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:12.328436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566910f-2b06-4674-9edb-474529a05170 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.328720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566910f-2b06-4674-9edb-474529a05170 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.329124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566910f-2b06-4674-9edb-474529a05170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.359100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3566910f-2b06-4674-9edb-474529a05170 None None] MFA Rules not processed for user `ee899d4497b5423c896fa3d1dfbbda2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:12.360443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3566910f-2b06-4674-9edb-474529a05170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.450745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 901/1813] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:12.459193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.461012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.536087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a4b5-224e-4b4c-a2db-aa7665665eb5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.540905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.541794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.542168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.542535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.562288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.563811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f446f6f-3309-4ab8-b18e-c12d8b94b863 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.575971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 913/1814] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:12.599509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566910f-2b06-4674-9edb-474529a05170 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.718238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566910f-2b06-4674-9edb-474529a05170 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:12.724208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:12.724861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.725139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.725400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] PATH_INFO: `/v3/users/ee899d4497b5423c896fa3d1dfbbda2c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.726546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee899d4497b5423c896fa3d1dfbbda2c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:12.728517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:12.796087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee899d4497b5423c896fa3d1dfbbda2c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:12.808959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee899d4497b5423c896fa3d1dfbbda2c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:12.809673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee899d4497b5423c896fa3d1dfbbda2c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:12.818176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee899d4497b5423c896fa3d1dfbbda2c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:12.818878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee899d4497b5423c896fa3d1dfbbda2c'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:12.844465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 902/1815] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:12 2022] DELETE /identity/v3/users/ee899d4497b5423c896fa3d1dfbbda2c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:12.846240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:12.847073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:12.847504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:12.872951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:12.874320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.949072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 914/1816] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:12.963567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:12.964947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:12.970422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:13.036395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0872e-2fe5-4051-9428-a38b4a346ab5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.040159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.040876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.041188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.041425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.050849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:13.052328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffc3fd-3fd7-46dc-b499-de4d96472999 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:13.074123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 903/1817] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:13.374356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:13.376290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:13.389283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086518a1-066a-4f41-b751-6adc73ce8063 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.395290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.395960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.396227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.396513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] PATH_INFO: `/v3/projects/d4deb529a89341d48d178161cc426865` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.397805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4deb529a89341d48d178161cc426865)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:13.399822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:13.417523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:13.459078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4deb529a89341d48d178161cc426865', 'request_id': 'req-49cb4362-9587-472c-904a-e52705823295'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:13.468550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4deb529a89341d48d178161cc426865', 'request_id': 'req-49cb4362-9587-472c-904a-e52705823295'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:13.479992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4deb529a89341d48d178161cc426865', 'request_id': 'req-49cb4362-9587-472c-904a-e52705823295'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:13.501906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] The token cache is being invalidate because project d4deb529a89341d48d178161cc426865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:13.502284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:13.503616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49cb4362-9587-472c-904a-e52705823295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:13.508691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 915/1818] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:13 2022] DELETE /identity/v3/projects/d4deb529a89341d48d178161cc426865 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:13.510849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.511335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.511446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.535181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:13.536266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:13.619370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 904/1819] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:13.624540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.625116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.626504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.658634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:13.660095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:13.668643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 916/1820] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:13.673288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:13.694598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085151-5589-4805-8e23-fb5c2f902e10 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.713266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.713859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.714138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.714410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.715079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:13.715425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:13.718715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:13.731792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 905/1821] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:13.736135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:13.746849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc56fcd-d619-487c-805c-5e931618a613 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.759222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.760224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.760566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.760903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.761601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:13.763657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:13.801327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 917/1822] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:13 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:13.824424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:13.836778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f840a5-7e96-4102-b481-8c8606b9c21e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.924345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.924932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.925717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.925995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.926707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:13.928819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:13.965373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 906/1823] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:13 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:13.969586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:13.985581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad78e42-cbbf-4d8f-932d-fe4256fc2829 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:13.995419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:13.996386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:13.996814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:13.997257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:13.998166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.000265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.019223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 918/1824] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:14.022614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.034369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5a11b-81d4-4279-8761-d93b5b7f2289 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.046263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.047617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.047910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.048363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.049528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.051563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572efb-a662-43d9-b87f-558650c8b0d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.083927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 907/1825] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:14.099808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.122546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7503-cecf-4438-8a3c-e11fe93e7d58 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.122546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.122933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.123258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.124836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] PATH_INFO: `/v3/projects/958a794c415743f2b259ae93b41e3e4a/users/c29f461191ed49fa84d745513048888a/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.151056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=958a794c415743f2b259ae93b41e3e4a, user_id=c29f461191ed49fa84d745513048888a, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.153924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.172179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 919/1826] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:14 2022] PUT /identity/v3/projects/958a794c415743f2b259ae93b41e3e4a/users/c29f461191ed49fa84d745513048888a/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:14.183700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.184328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.184789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.213546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:14.226670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] MFA Rules not processed for user `c29f461191ed49fa84d745513048888a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:14.228037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:14.350639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 908/1827] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:14.352258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.457806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904919-8ba9-4a8c-80af-f51bcbb4d0dd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.463495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.464365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.464802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.465079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.465740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.467858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.499184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 920/1828] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:14 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:14.501827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.513594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268b30a-12bc-4976-81f2-f6daca42bf21 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.576148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.576766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.577128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.577395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.578055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.580095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.608057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 909/1829] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:14 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:14.609501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.623707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797fbff-990d-4935-beec-d363d86ac7b0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.629697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.630323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.630603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.630874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.631620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.633231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.645170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 921/1830] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:14.649302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.659999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9099c9a-e7ea-47df-9620-55b9da63237f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.664918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.665580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.665865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.666128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.666777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.668454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2646b1-634f-418e-95c0-8c453a07a664 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.679084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 910/1831] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:14.683131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.693617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e841c-1351-4382-a71f-c38012dec381 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.698462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.699140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.699470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.699771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] PATH_INFO: `/v3/projects/4ccd62e38ece475ca70e4d80b0b65328/users/07a36d347af74735b8f729ba0675444a/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.716058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:14.723530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccd62e38ece475ca70e4d80b0b65328, user_id=07a36d347af74735b8f729ba0675444a, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.727040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:14.749288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 922/1832] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:14 2022] PUT /identity/v3/projects/4ccd62e38ece475ca70e4d80b0b65328/users/07a36d347af74735b8f729ba0675444a/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:14.759476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.759761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.760039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.789992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] MFA Rules not processed for user `07a36d347af74735b8f729ba0675444a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:14.791261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:14.865026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.867328 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:14.901106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 911/1833] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:14.905552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:14.925919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:14.972044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72442-343f-40b8-b370-9acd567ea6de admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:14.982915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:14.984342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:14.984842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:14.985345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] PATH_INFO: `/v3/users/9dc5e8a8fc984a22bf3ad3bf743ba2db` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:14.986916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc5e8a8fc984a22bf3ad3bf743ba2db)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:14.990702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.009087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090e7b7-c71a-4f38-b4ef-e8e8faf6aff1 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.020282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.020921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.021444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.021990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.022632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.025353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.073626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc5e8a8fc984a22bf3ad3bf743ba2db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.093771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc5e8a8fc984a22bf3ad3bf743ba2db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.098135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 912/1834] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:14 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:15.100160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.101535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc5e8a8fc984a22bf3ad3bf743ba2db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.102230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc5e8a8fc984a22bf3ad3bf743ba2db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.102904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc5e8a8fc984a22bf3ad3bf743ba2db'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.103835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.160283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 923/1835] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:14 2022] DELETE /identity/v3/users/9dc5e8a8fc984a22bf3ad3bf743ba2db => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:15.163973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.168372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.235535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446dd49-20c3-4a9e-b925-49578760bdde admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.314612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a462-66c9-46d5-b42d-28d7cd5ba984 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.319325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.320261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.320455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.320843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.321253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.321460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] PATH_INFO: `/v3/users/15e13d02aaff40e0b1bf83dfd1fcb6df` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.321661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.322000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.323112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.323703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e13d02aaff40e0b1bf83dfd1fcb6df)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.325935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.327548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de905c-eaaa-4e29-860a-814714c0a045 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.355119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:15.389725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 924/1836] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:15 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:15.389725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.391096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.391399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.399241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:15.407352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e13d02aaff40e0b1bf83dfd1fcb6df'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.426140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e13d02aaff40e0b1bf83dfd1fcb6df'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.426936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e13d02aaff40e0b1bf83dfd1fcb6df'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.437763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:15.441214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e13d02aaff40e0b1bf83dfd1fcb6df'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.441362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.441988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e13d02aaff40e0b1bf83dfd1fcb6df'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:15.453273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 925/1837] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:15.455277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.463609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 913/1838] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:15 2022] DELETE /identity/v3/users/15e13d02aaff40e0b1bf83dfd1fcb6df => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:15.468345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.468540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.470098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.482984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994aca2-75d2-47a2-80bf-a30db301cba7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.563421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428224f8-d3ac-4c47-a0eb-4b2891e1191d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.568305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.569124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.569527 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.569888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.592573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.593275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.593604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.593962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.594681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.598323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f974580-1dfc-4a58-bec8-c825364e7afe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.613910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 926/1839] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:15.613910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.613910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.613910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.639334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:15.640790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:15.664496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 927/1840] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:15.667889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.674321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.677326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.684824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02708b9-f7cd-4ef2-98d6-b1fdfcaae176 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.690526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.691492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.691492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.691703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] PATH_INFO: `/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff/users/78966b244dd242fe931653deab0ee326/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.699598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 914/1841] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:15.700941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.709352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b242860-2314-4c08-b22d-7e10e2b18cbd service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.713212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.713924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.714218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.714475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.726027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.727425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.739304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 915/1842] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:15.749498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c7f6f7b604e5fa899430930ac17ff, user_id=78966b244dd242fe931653deab0ee326, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.752263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.772380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 928/1843] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:15 2022] PUT /identity/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff/users/78966b244dd242fe931653deab0ee326/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:15.776418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.789463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49418aa-94a4-4c42-9486-8f3a70e6be78 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.868173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:15.886351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.887061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.887364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.887633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.888293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.890797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49910ed2-ec16-432d-91d8-5b9a19309dc1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:15.912575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 916/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:15.914974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:15.951309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dcb84-e452-473c-b171-2ad882f00bf5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:15.967192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:15.968977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:15.969138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:15.970584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] PATH_INFO: `/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff/users/78966b244dd242fe931653deab0ee326/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:15.975411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c7f6f7b604e5fa899430930ac17ff, user_id=78966b244dd242fe931653deab0ee326, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:15.981454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:16.015230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 929/1845] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:15 2022] PUT /identity/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff/users/78966b244dd242fe931653deab0ee326/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:16.020032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:16.020441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:16.020816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:16.067423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] MFA Rules not processed for user `78966b244dd242fe931653deab0ee326`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:16.068927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:16.241229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 917/1846] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:16.263377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:16.265315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:16.275999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:16.282062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:16.361398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c68369-1782-4ab7-bf93-28a4469f890e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:16.372156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:16.374964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:16.375411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:16.375788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:16.380237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:16.387439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81d521-97f7-4cc9-baf1-738fc61c870d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:16.392820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:16.394441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:16.394441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:16.394441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] PATH_INFO: `/v3/projects/a748e37ed7bc4fe3a91fc69b281236fa` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:16.395785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a748e37ed7bc4fe3a91fc69b281236fa)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:16.397853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:16.465795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:16.467402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:16.468906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:16.473553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a748e37ed7bc4fe3a91fc69b281236fa', 'request_id': 'req-93a91d51-b461-4029-b81d-f1246101201b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.492651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a748e37ed7bc4fe3a91fc69b281236fa', 'request_id': 'req-93a91d51-b461-4029-b81d-f1246101201b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.494458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 918/1847] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:16.525412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a748e37ed7bc4fe3a91fc69b281236fa', 'request_id': 'req-93a91d51-b461-4029-b81d-f1246101201b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.555825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] The token cache is being invalidate because project a748e37ed7bc4fe3a91fc69b281236fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:16.558378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.559819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.563806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 930/1848] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:16 2022] DELETE /identity/v3/projects/a748e37ed7bc4fe3a91fc69b281236fa => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:16.568217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:16.589571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d5da5-e3dc-421c-810a-c642641411be service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:16.652060 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:16.654911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:16.671895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:16.745923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:16.747546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:16.748163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:16.748448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] PATH_INFO: `/v3/users/0d7a74b9c5304afdb48227a23244ddac` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:16.752473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7a74b9c5304afdb48227a23244ddac)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:16.757560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:16.829950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a91d51-b461-4029-b81d-f1246101201b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:16.843013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7a74b9c5304afdb48227a23244ddac'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.844579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:16.845305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:16.845584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:16.845852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] PATH_INFO: `/v3/projects/d74aba013735417b907c9bfd7893b3e3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:16.847439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74aba013735417b907c9bfd7893b3e3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:16.850646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:16.877531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7a74b9c5304afdb48227a23244ddac'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.878623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7a74b9c5304afdb48227a23244ddac'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.886678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7a74b9c5304afdb48227a23244ddac'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.887672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7a74b9c5304afdb48227a23244ddac'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.912809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 919/1849] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:16 2022] DELETE /identity/v3/users/0d7a74b9c5304afdb48227a23244ddac => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:16.949546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74aba013735417b907c9bfd7893b3e3', 'request_id': 'req-631e1665-f54e-45c3-884f-91b5adb3f903'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.961764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74aba013735417b907c9bfd7893b3e3', 'request_id': 'req-631e1665-f54e-45c3-884f-91b5adb3f903'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.973950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74aba013735417b907c9bfd7893b3e3', 'request_id': 'req-631e1665-f54e-45c3-884f-91b5adb3f903'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.988790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] The token cache is being invalidate because project d74aba013735417b907c9bfd7893b3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:16.988790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.990872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:16.994222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 931/1850] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:16 2022] DELETE /identity/v3/projects/d74aba013735417b907c9bfd7893b3e3 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:16.998788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:17.003775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:17.114004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827afac9-955f-4c76-8ab5-623e7b649f77 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:17.222925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:17.223727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:17.224151 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:17.224634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] PATH_INFO: `/v3/users/be9991b4242d4195b9a73fb910a4aa55` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:17.227201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9991b4242d4195b9a73fb910a4aa55)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:17.229612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:17.232430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:17.321050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9991b4242d4195b9a73fb910a4aa55'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.350362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9991b4242d4195b9a73fb910a4aa55'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.351594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9991b4242d4195b9a73fb910a4aa55'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.366726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9991b4242d4195b9a73fb910a4aa55'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.368545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9991b4242d4195b9a73fb910a4aa55'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.404296 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 920/1851] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:16 2022] DELETE /identity/v3/users/be9991b4242d4195b9a73fb910a4aa55 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:17.655402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:17.656855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:17.680431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e1665-f54e-45c3-884f-91b5adb3f903 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:17.751966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:17.753483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:17.790585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:17.793082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:17.793431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:17.794431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] PATH_INFO: `/v3/projects/834e0a3ea43042fb9c98d9503ffa3533` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:17.796112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834e0a3ea43042fb9c98d9503ffa3533)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:17.799324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:17.809096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:17.850931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce8659-934a-423e-a418-f33c9d64aa85 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:17.861554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:17.863201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:17.864060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:17.865862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] PATH_INFO: `/v3/users/a2536bc7c170420da6e107d012555818` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:17.868978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2536bc7c170420da6e107d012555818)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:17.869721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834e0a3ea43042fb9c98d9503ffa3533', 'request_id': 'req-dde2fac0-e423-419d-b41c-0a1ca10ab103'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.874613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:17.877484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834e0a3ea43042fb9c98d9503ffa3533', 'request_id': 'req-dde2fac0-e423-419d-b41c-0a1ca10ab103'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.887858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834e0a3ea43042fb9c98d9503ffa3533', 'request_id': 'req-dde2fac0-e423-419d-b41c-0a1ca10ab103'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.912965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] The token cache is being invalidate because project 834e0a3ea43042fb9c98d9503ffa3533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:17.913322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.913891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.919128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 932/1852] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:17 2022] DELETE /identity/v3/projects/834e0a3ea43042fb9c98d9503ffa3533 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:17.930074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:17.931777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:17.982729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2536bc7c170420da6e107d012555818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.997346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2536bc7c170420da6e107d012555818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:17.998090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2536bc7c170420da6e107d012555818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.005865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2536bc7c170420da6e107d012555818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.006514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6fc52031-717e-40db-8273-b47da6d022ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2536bc7c170420da6e107d012555818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.012800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2fac0-e423-419d-b41c-0a1ca10ab103 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.017716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:18.021666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.022330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.022616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.022951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] PATH_INFO: `/v3/projects/345ce7ea557b458891b71b83db65c75c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.024400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345ce7ea557b458891b71b83db65c75c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.024572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 921/1853] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:17 2022] DELETE /identity/v3/users/a2536bc7c170420da6e107d012555818 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:18.026210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b0dd-d2ff-49c4-9189-ee9e4115ba74 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.026619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b0dd-d2ff-49c4-9189-ee9e4115ba74 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.027181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.027464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b0dd-d2ff-49c4-9189-ee9e4115ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.051668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9e47b0dd-d2ff-49c4-9189-ee9e4115ba74 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:18.053165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9e47b0dd-d2ff-49c4-9189-ee9e4115ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.082504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345ce7ea557b458891b71b83db65c75c', 'request_id': 'req-10762e02-290e-424d-a680-7ab6033081e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.090526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345ce7ea557b458891b71b83db65c75c', 'request_id': 'req-10762e02-290e-424d-a680-7ab6033081e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.100157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345ce7ea557b458891b71b83db65c75c', 'request_id': 'req-10762e02-290e-424d-a680-7ab6033081e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.111887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] The token cache is being invalidate because project 345ce7ea557b458891b71b83db65c75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:18.112165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.112813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-10762e02-290e-424d-a680-7ab6033081e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.114627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 933/1854] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:17 2022] DELETE /identity/v3/projects/345ce7ea557b458891b71b83db65c75c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:18.115871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6877d6-188b-4ec0-ae6c-66d201d592ca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.116148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6877d6-188b-4ec0-ae6c-66d201d592ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.116414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6877d6-188b-4ec0-ae6c-66d201d592ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.122352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 922/1855] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:18.124486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b4a35-6ef8-4ce7-b19d-6131dd096c88 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.124774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b4a35-6ef8-4ce7-b19d-6131dd096c88 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.125074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b4a35-6ef8-4ce7-b19d-6131dd096c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.148243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2a6877d6-188b-4ec0-ae6c-66d201d592ca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:18.149205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9f1b4a35-6ef8-4ce7-b19d-6131dd096c88 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:18.149455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a6877d6-188b-4ec0-ae6c-66d201d592ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.150516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9f1b4a35-6ef8-4ce7-b19d-6131dd096c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.170877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:18.233611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 934/1856] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:18.235521 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 923/1857] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:18.238010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.238296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.238567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.239219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.239423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.239707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.259307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:18.260591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.273235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 935/1858] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:18.273235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.273235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.274542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:18.275800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.285546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 924/1859] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:18.287639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.300945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32508326-c53c-4052-9b79-611f9b28fd9e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.305873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.306483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.306763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.307034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.307723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:18.308026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.309812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.322775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 925/1860] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:18.323673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.335703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cabdd-1caf-45bd-a086-e163dc1f660e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.340595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.341589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.341907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.342182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.342918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:18.343254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.344872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.355262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648674a0-7fdc-4fd1-b3ac-0a6cf3faf78e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.359189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 926/1861] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:18.359189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.359392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.359725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.360001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.360269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.368687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cfb18-9a96-4726-acf0-dc7b1457bf74 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.369974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.371450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.373392 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.374048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.374335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.374591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.375250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.377164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.382755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 936/1862] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:18.383759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.385516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.408643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 927/1863] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:18.409777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.455609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6b04-ee46-4125-9d27-7df25e1b0e21 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.466502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:18.485434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911824c-fd97-4133-8b4e-3b867ccf341d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.490808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.491608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.491958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.492299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.493037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.495063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.511486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.512277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.512584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.512942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.514999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.533277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 928/1864] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:18.533277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.559182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:18.595719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.597166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.609506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 937/1865] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:18.610164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.639838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487315-1041-419a-bf6c-62b9fda37eff service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.644567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83037c-28a5-4f92-8e50-597fca076de5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.645087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.645706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.646243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.646243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.646869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.648849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.651999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.652928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.653961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.654390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.655258 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.658126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.682346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 938/1866] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/users => generated 292 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:18.683659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.697618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 929/1867] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:18 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:18.701531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4fcc9-9e1a-42d9-b782-de035888c646 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.702371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.714462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.715304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.715587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.715852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.716569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.717601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466b235-21dd-4299-9f0d-6e67a97acb30 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.724947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.725589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.726229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.726502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.727319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.728967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.729943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.740268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 930/1868] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:18.742901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 939/1869] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:18.746764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.746904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.749278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.761091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78d1cf-117a-4d2e-8023-760200f155f8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.763029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7f0b2-5903-40fa-8b39-84e0abec4f52 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.768267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.768557 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.768955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.769191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.769302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.769465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.769564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.769732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] PATH_INFO: `/v3/projects/20242d235619491d8f622f01699e9f5c` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.770254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.770849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] RBAC: Authorizing `identity:delete_project(project_id=20242d235619491d8f622f01699e9f5c)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.771977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.772802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.785563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 940/1870] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:18.786524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.796627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e3851-2979-4f99-acb2-85d64b6713c1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.801797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.802376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.802742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.803061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.803818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.805503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.819200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 941/1871] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:18.819200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.827193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd876c-ea49-4e1b-ad95-8c51da6da866 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.829558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20242d235619491d8f622f01699e9f5c', 'request_id': 'req-661a4eaf-a070-4047-96ac-a172f90bafca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.838791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20242d235619491d8f622f01699e9f5c', 'request_id': 'req-661a4eaf-a070-4047-96ac-a172f90bafca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.849220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20242d235619491d8f622f01699e9f5c', 'request_id': 'req-661a4eaf-a070-4047-96ac-a172f90bafca'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.864372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] The token cache is being invalidate because project 20242d235619491d8f622f01699e9f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:18.864993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.865728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:18.867360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 931/1872] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:18 2022] DELETE /identity/v3/projects/20242d235619491d8f622f01699e9f5c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:18.868101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.869414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:18.902012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:18.902894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:18.903104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:18.903443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] PATH_INFO: `/v3/projects/1fc0e5b9e10c496985063b8a53f5501a/users/532119fdb1994e5b9f1a073a9c300381/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:18.907756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:18.923416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc0e5b9e10c496985063b8a53f5501a, user_id=532119fdb1994e5b9f1a073a9c300381, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:18.926984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:18.942476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 942/1873] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:18 2022] PUT /identity/v3/projects/1fc0e5b9e10c496985063b8a53f5501a/users/532119fdb1994e5b9f1a073a9c300381/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:18.943415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:18.943611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a4eaf-a070-4047-96ac-a172f90bafca admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:18.945045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.021219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.023273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.023558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.023832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] PATH_INFO: `/v3/projects/43bc7cf474fa4518bc9aa3102f707550/users/335ed1e0f6dc4c9f844c5cebdcbcc5dd/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.029202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8aaa0-ddde-4329-abe6-e366d876b1b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.039685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.040873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.041013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc7cf474fa4518bc9aa3102f707550, user_id=335ed1e0f6dc4c9f844c5cebdcbcc5dd, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.041449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.041815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] PATH_INFO: `/v3/users/891505a969844c6881c0d4598ec6f1a3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.043253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] RBAC: Authorizing `identity:delete_user(user_id=891505a969844c6881c0d4598ec6f1a3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.043835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f1663-c9d9-4867-baf2-6d8074f6ad3c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.045715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.054175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:19.066385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 932/1874] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:18 2022] PUT /identity/v3/projects/43bc7cf474fa4518bc9aa3102f707550/users/335ed1e0f6dc4c9f844c5cebdcbcc5dd/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:19.068116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.068402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.068779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.105760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] MFA Rules not processed for user `532119fdb1994e5b9f1a073a9c300381`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:19.107013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.154958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891505a969844c6881c0d4598ec6f1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.169897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891505a969844c6881c0d4598ec6f1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.183678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891505a969844c6881c0d4598ec6f1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.184774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891505a969844c6881c0d4598ec6f1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.185557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ae9a97e3-4df9-4b66-8a61-572d6b3f234c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891505a969844c6881c0d4598ec6f1a3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.207892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 943/1875] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:18 2022] DELETE /identity/v3/users/891505a969844c6881c0d4598ec6f1a3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:19.213705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.214198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.214934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.227742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 933/1876] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:19.242310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:19.243946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.260486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] MFA Rules not processed for user `335ed1e0f6dc4c9f844c5cebdcbcc5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:19.262531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.357414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 944/1877] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:19.358720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:19.360898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.380893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd732ce-21b4-4139-b80e-65e00465917b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.388415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:19.446786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca573e75-b21a-4695-9f18-cd186a1be63f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.454940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.455886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.456198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.456475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.459103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.471265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.472312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.472681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.472992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.500656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:19.545325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.546844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.548582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.550024 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.558391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 934/1878] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:19.560209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:19.562120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.562560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 945/1879] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:19.580861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:19.582841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.715178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593a7e0-0ea1-4169-977f-0f906293fa06 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.715178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.715178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.715178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.715583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.722644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.722644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.725100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfdac-b572-4d3e-9623-00426871b6bb service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.731327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.732538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 935/1880] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:19.732660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.735426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.735739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] PATH_INFO: `/v3/users/532119fdb1994e5b9f1a073a9c300381` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.737087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=532119fdb1994e5b9f1a073a9c300381)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.739106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.788262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:19.791606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.812497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e4e1a-c3d0-4fbf-95aa-484cec017427 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:19.815521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532119fdb1994e5b9f1a073a9c300381'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.819697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:19.820246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.820524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.820931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] PATH_INFO: `/v3/projects/174bff59f4bd4c018f6ab99c188552ca` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.822088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=174bff59f4bd4c018f6ab99c188552ca)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:19.824925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:19.843461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532119fdb1994e5b9f1a073a9c300381'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.854808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532119fdb1994e5b9f1a073a9c300381'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.855549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532119fdb1994e5b9f1a073a9c300381'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.856151 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8ec15eac-42cd-4066-8e31-e4f5dfaa14fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532119fdb1994e5b9f1a073a9c300381'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.877624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 946/1881] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:19 2022] DELETE /identity/v3/users/532119fdb1994e5b9f1a073a9c300381 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:19.882254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:19.882539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:19.882939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:19.901777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:19.920984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:19.922320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:19.929899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174bff59f4bd4c018f6ab99c188552ca', 'request_id': 'req-9468da7d-5a5f-48cc-9352-632c910298fd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.946624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174bff59f4bd4c018f6ab99c188552ca', 'request_id': 'req-9468da7d-5a5f-48cc-9352-632c910298fd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.966885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174bff59f4bd4c018f6ab99c188552ca', 'request_id': 'req-9468da7d-5a5f-48cc-9352-632c910298fd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.985114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:19.989388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] The token cache is being invalidate because project 174bff59f4bd4c018f6ab99c188552ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:19.989657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.992171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9468da7d-5a5f-48cc-9352-632c910298fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:19.999550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 936/1882] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:19 2022] DELETE /identity/v3/projects/174bff59f4bd4c018f6ab99c188552ca => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:20.007595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cf8f-cc61-46bc-9e4d-0ade4d942c5a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.008070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cf8f-cc61-46bc-9e4d-0ade4d942c5a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.008352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cf8f-cc61-46bc-9e4d-0ade4d942c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.023063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 947/1883] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:20.033520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a9e0cf8f-cc61-46bc-9e4d-0ade4d942c5a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:20.033804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.036004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.036341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9e0cf8f-cc61-46bc-9e4d-0ade4d942c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.051204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 937/1884] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:20.057189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.057508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.057808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.079068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:20.080450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.097170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 938/1885] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:20.101075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.112648 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b518f-cfe0-4230-9631-d202336dcb23 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.124332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.124966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.125246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.126351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.126351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:20.126556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.128967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.133834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205c5d-eb5d-44c8-987d-89eabddc9c84 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.140988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.141614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.141877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.142383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.144373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 939/1886] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:20.152717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.153183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.154359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.164906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8c9db-b6b8-4a32-b2fc-a1a5b134c0ce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.168464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 948/1887] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:20.172835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.173897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.174457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.174963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.175851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.178182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.226628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 940/1888] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:20.230852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.243723 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51d39-d8c6-4273-9689-546cf1d09303 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.331588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.331588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.331588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.331588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.332318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.334950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.369506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 949/1889] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:20.373684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.390311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff29a-de9d-435c-b5e6-eb3288334155 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.391162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.391859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.392226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.392570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.393373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.395749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.407732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 941/1890] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:20.412702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.426578 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649e161-529d-4954-9df6-0a6750a4743f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.440674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.441495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.441731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.442109 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.442631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.444266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87fef4-0e1d-400e-a885-59ed6b955da2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.457385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 950/1891] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:20.461693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.473572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d938eb9-68e7-4525-9fde-099535e1b21d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.478429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.479101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.479548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.479752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] PATH_INFO: `/v3/projects/44f36c0fd60f4665be6d117d4e6c88b4/users/f5780dce7aca4b5fba609a2a4d938fa9/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.495253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f36c0fd60f4665be6d117d4e6c88b4, user_id=f5780dce7aca4b5fba609a2a4d938fa9, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.497974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.512061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 942/1892] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:20 2022] PUT /identity/v3/projects/44f36c0fd60f4665be6d117d4e6c88b4/users/f5780dce7aca4b5fba609a2a4d938fa9/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:20.523865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.524157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.524434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.557207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] MFA Rules not processed for user `f5780dce7aca4b5fba609a2a4d938fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:20.558529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.642005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:20.659477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 951/1893] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:20.660696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.683037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.685235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.816125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b675e8-2f05-4edd-a586-633dd793a798 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.820122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e738f39-f6a8-415d-882e-9869105d61ca admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:20.825035 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.826585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.826880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.828282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] PATH_INFO: `/v3/users/2774a77cb0e54f64b854d0aff9c7d270` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.829561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2774a77cb0e54f64b854d0aff9c7d270)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.831724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.834099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:20.834728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:20.835007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:20.835315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:20.836086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:20.838491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:20.880886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 943/1894] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:20.884965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.887114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.894100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2774a77cb0e54f64b854d0aff9c7d270'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:20.908530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2774a77cb0e54f64b854d0aff9c7d270'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:20.917015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2774a77cb0e54f64b854d0aff9c7d270'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:20.917629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2774a77cb0e54f64b854d0aff9c7d270'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:20.918454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2774a77cb0e54f64b854d0aff9c7d270'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:20.921301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:20.936587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 952/1895] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:20 2022] DELETE /identity/v3/users/2774a77cb0e54f64b854d0aff9c7d270 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:20.939914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:20.944156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:20.962359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf251de-a122-49c3-bf93-a012ecb40d31 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.064771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70f45d-23d6-4421-afc0-e250696ec0bd admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.071597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.072634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.073175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.073658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.073778 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.074411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.074694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.074968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] PATH_INFO: `/v3/projects/1fc0e5b9e10c496985063b8a53f5501a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.075407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.078401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.082848 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc0e5b9e10c496985063b8a53f5501a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.087641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.122344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 953/1896] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:20 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:21.124963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.134528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:21.141048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194eec11-2592-4a48-b6cb-24df3bc71744 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.152444 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.154007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.162312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.163572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.164395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.166308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.180855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 954/1897] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:21.185093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.199595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96505b93-8096-49d6-b38c-d7aa3b291862 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.213345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc0e5b9e10c496985063b8a53f5501a', 'request_id': 'req-67c90d49-0ad3-4640-ab70-4465e4fa2668'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.233383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc0e5b9e10c496985063b8a53f5501a', 'request_id': 'req-67c90d49-0ad3-4640-ab70-4465e4fa2668'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.242481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc0e5b9e10c496985063b8a53f5501a', 'request_id': 'req-67c90d49-0ad3-4640-ab70-4465e4fa2668'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.256063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] The token cache is being invalidate because project 1fc0e5b9e10c496985063b8a53f5501a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:21.256356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.257037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.258816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 944/1898] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:20 2022] DELETE /identity/v3/projects/1fc0e5b9e10c496985063b8a53f5501a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:21.259563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.260857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.277776 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.278452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.278739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.279002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] PATH_INFO: `/v3/projects/ddea359807ad4437a531b7a08a7dd20d/users/0df203ee6464493590bda822473b94b6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.293932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddea359807ad4437a531b7a08a7dd20d, user_id=0df203ee6464493590bda822473b94b6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.296787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef2af-a0a0-4fdb-bd41-0b824c0558fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.311527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 955/1899] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:21 2022] PUT /identity/v3/projects/ddea359807ad4437a531b7a08a7dd20d/users/0df203ee6464493590bda822473b94b6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:21.316044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.316331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.316601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.332316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90d49-0ad3-4640-ab70-4465e4fa2668 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.337202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:21.338457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.341502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:21.399045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.399884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.400165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.400436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] PATH_INFO: `/v3/projects/11fcf369aff04afca1e501f8714117af` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.401583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fcf369aff04afca1e501f8714117af)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.403319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 956/1900] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:21.403637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.408118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.410227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.423032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df820778-f10c-4370-a56c-aaf6666c508b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.429737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.430724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.430865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.431455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.432527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.434618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9de22-fe01-4f4e-a3f3-310a79147b5b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.448614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 957/1901] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:21.448614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.448614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.448614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.465760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fcf369aff04afca1e501f8714117af', 'request_id': 'req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.468364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:21.469579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.472360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fcf369aff04afca1e501f8714117af', 'request_id': 'req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.481266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fcf369aff04afca1e501f8714117af', 'request_id': 'req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.492943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] The token cache is being invalidate because project 11fcf369aff04afca1e501f8714117af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:21.493231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.493872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.495476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 945/1902] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:21 2022] DELETE /identity/v3/projects/11fcf369aff04afca1e501f8714117af => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:21.496312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.499229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.529560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 958/1903] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:21.530519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.531825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.539899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:21.543610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ed5d0-6251-493d-bbf8-3f6006efcd82 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.551382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.552021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.552442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.552710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] PATH_INFO: `/v3/users/935e7449ff544b2db80b7837d702f470` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.553869 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=935e7449ff544b2db80b7837d702f470)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.555834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.567942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df775fe-ce3a-4443-9b15-c6a12cf68e0f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.573105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.573734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.574011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.574283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] PATH_INFO: `/v3/projects/ddea359807ad4437a531b7a08a7dd20d/users/0df203ee6464493590bda822473b94b6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.576341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddea359807ad4437a531b7a08a7dd20d, user_id=0df203ee6464493590bda822473b94b6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.579022 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.590590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 946/1904] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:21 2022] PUT /identity/v3/projects/ddea359807ad4437a531b7a08a7dd20d/users/0df203ee6464493590bda822473b94b6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:21.591412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.620697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935e7449ff544b2db80b7837d702f470'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.634880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935e7449ff544b2db80b7837d702f470'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.641982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935e7449ff544b2db80b7837d702f470'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.642720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935e7449ff544b2db80b7837d702f470'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.643399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-49223fb5-294b-4b49-a1ed-6307cd8de94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935e7449ff544b2db80b7837d702f470'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:21.645999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:21.658279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 959/1905] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:21 2022] DELETE /identity/v3/users/935e7449ff544b2db80b7837d702f470 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:21.659907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.660194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.660470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.670240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e534e-5743-441b-abd2-0e3a3f20e73a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.693090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] MFA Rules not processed for user `0df203ee6464493590bda822473b94b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:21.694390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.741687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.742432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.742713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.743511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.744173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:21.744712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.746506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.760840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 947/1906] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:21.768149 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.775303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.779788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 960/1907] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:21.793484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.795044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:21.796407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08144-f073-4b65-b8a9-1f9b14096846 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.801733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.802681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.803391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.803391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:21.804769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:21.807130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:21.855891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 948/1908] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:21 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:21.856902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:21.902989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc3ca-d73f-403c-a8c0-c2e3262ae4f8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.920275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:21.986188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8988f-726b-4f3a-8041-b980fe1505e1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:21.986696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:21.987340 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:21.987615 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:21.987881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.001130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.001827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.002167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.002498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.003268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.005466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.032290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 949/1909] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:21 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:22.036257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.039483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.051808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afe177-2fab-4097-b115-d9cee082d2eb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.056781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.057406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.057680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.057949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] PATH_INFO: `/v3/projects/a156c390c144446280de6920e45ca855` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.059046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a156c390c144446280de6920e45ca855)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.060992 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.070014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.071617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.083369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 961/1910] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:22.084037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.096385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30457-af8f-4ebb-b91a-f099807a22c8 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.101144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.101768 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.102042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.102316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.102967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.104618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.114942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:22.115218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 962/1911] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:22.116405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.118424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.124126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a156c390c144446280de6920e45ca855', 'request_id': 'req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.130956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd4daf-ecd3-47dc-8e4e-ed16eb3b6bb9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.131833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a156c390c144446280de6920e45ca855', 'request_id': 'req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.140619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a156c390c144446280de6920e45ca855', 'request_id': 'req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.151423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] The token cache is being invalidate because project a156c390c144446280de6920e45ca855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:22.151788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.152510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.154125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 950/1912] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:22 2022] DELETE /identity/v3/projects/a156c390c144446280de6920e45ca855 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:22.155081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.157818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.195553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.196221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.196617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.196902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] PATH_INFO: `/v3/users/f5780dce7aca4b5fba609a2a4d938fa9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.198157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5780dce7aca4b5fba609a2a4d938fa9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.200138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.230950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6411-fa32-43d6-ad4b-95b92f3b785b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.236704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.237582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.237933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.238295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.240491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.242257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c77fb-c37f-4bf8-afaa-1cfc1a0df625 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.253419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 951/1913] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:22.254704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.254991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.255295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.261705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5780dce7aca4b5fba609a2a4d938fa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.270953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:22.277122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:22.278424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.287699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5780dce7aca4b5fba609a2a4d938fa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.292178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 952/1914] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:22.292974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.296068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5780dce7aca4b5fba609a2a4d938fa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.299559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5780dce7aca4b5fba609a2a4d938fa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.301075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9e45c65b-9d49-4f4a-bea1-e747c21846c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5780dce7aca4b5fba609a2a4d938fa9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:22.319653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df528e-08f8-434d-b0c5-82d33fd60b58 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.322292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 963/1915] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:22 2022] DELETE /identity/v3/users/f5780dce7aca4b5fba609a2a4d938fa9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:22.327067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9d7e-9c9c-4ab2-b399-d427f0384868 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.327430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9d7e-9c9c-4ab2-b399-d427f0384868 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.327808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9d7e-9c9c-4ab2-b399-d427f0384868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.352207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-070c9d7e-9c9c-4ab2-b399-d427f0384868 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:22.353712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-070c9d7e-9c9c-4ab2-b399-d427f0384868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.428640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.429464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.429699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.430718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] PATH_INFO: `/v3/projects/c7dd2666ad8c417b9a76b0ec4112d908/users/0205327d17d04f15955792630eac762a/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.451643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7dd2666ad8c417b9a76b0ec4112d908, user_id=0205327d17d04f15955792630eac762a, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.454673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.461630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 964/1916] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:22.463420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc776493-0245-4f77-a997-5bf573cfa530 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.463778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc776493-0245-4f77-a997-5bf573cfa530 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.464154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc776493-0245-4f77-a997-5bf573cfa530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.485392 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 953/1917] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:22 2022] PUT /identity/v3/projects/c7dd2666ad8c417b9a76b0ec4112d908/users/0205327d17d04f15955792630eac762a/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:22.486243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.494769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dc776493-0245-4f77-a997-5bf573cfa530 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:22.496195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dc776493-0245-4f77-a997-5bf573cfa530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.568205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:22.585244 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 965/1918] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:22.586681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.587063 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.587434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.596189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62221-9194-43e5-842c-5a339086a89d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.599863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.600599 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.600878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.601254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.601925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:22.602263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.603960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.618547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 954/1919] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:22.619520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.621731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.624171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] MFA Rules not processed for user `0205327d17d04f15955792630eac762a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:22.626892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.694994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c3b3b-1556-448a-9ea2-bd25b5349c40 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.698618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.699256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.699535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.699805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.703635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 966/1920] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:22.704607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.712911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.714343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.715759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1d0d1-bf14-41c6-ac14-9cef54fba471 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.720582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.721174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.721454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.721756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.722389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.724316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.724702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 955/1921] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:22.725382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.726762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:22.729674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:22.740057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b84cf6-8c5a-4cf6-a930-fc8a461b6fd2 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.745616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.746273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.746526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.746887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.747559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.749937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.758480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 967/1922] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:22.762907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.782324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 956/1923] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:22.786212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.849977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db095d-2ae2-4a1d-9910-acea4e309d78 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.921107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.921742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.922020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.922289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.923078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.924777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93fe89-6b7f-4cae-9283-46ef1fe74f01 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.925237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.939469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.941714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.942181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.943518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.945837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.950330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:22.953918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 968/1924] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:22.955270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:22.983648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f050-c7f2-416d-a5e5-682a490db3c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:22.989425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:22.990410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:22.990642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:22.991006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:22.991846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:22.993678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.002115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 957/1925] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:22 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:23.005219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.010584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 969/1926] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:23.015080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.022260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ba2e1-e588-4b6c-ab7f-372258bd91ca admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.029146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.030192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35cbd6-e8fa-4eb0-a6c9-8c5274f47215 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.030393 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.030689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.030997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.031755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.034150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.037426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.038955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.039407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.039748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.041174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.042839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.048081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 958/1927] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:23.050976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.064927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d844322-2f3e-43da-a35f-b4fa06f539cb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.076890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 970/1928] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:23.081779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.092212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.093096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.093484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.093888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.097772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.097772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.106897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913dc7f-f301-4e5e-8c0e-7af6b2d87369 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.109470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 959/1929] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:23.113481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.117955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.119080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.119519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.120289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] PATH_INFO: `/v3/projects/9f27a977e86a41f3be77b9acf324ef99/users/d54dbe457d64489f9f55abe54bd72bec/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.132129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abcc55-8062-4cf3-9102-78135f7ecd16 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.144299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f27a977e86a41f3be77b9acf324ef99, user_id=d54dbe457d64489f9f55abe54bd72bec, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.147399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.149482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.150237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.151468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.151468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] PATH_INFO: `/v3/projects/bec34418a2a94c278cab0ae915c4ec69/users/1e185294e40a49179464b52e30091739/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.167528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 971/1930] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:23 2022] PUT /identity/v3/projects/9f27a977e86a41f3be77b9acf324ef99/users/d54dbe457d64489f9f55abe54bd72bec/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:23.168977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.180278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.189933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec34418a2a94c278cab0ae915c4ec69, user_id=1e185294e40a49179464b52e30091739, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.193210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15724-3a70-488c-9748-1b275f874c5d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.206178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:23.214584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 960/1931] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:23 2022] PUT /identity/v3/projects/bec34418a2a94c278cab0ae915c4ec69/users/1e185294e40a49179464b52e30091739/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:23.217802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba202-592a-4e05-b20b-4e1825236d83 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.219551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba202-592a-4e05-b20b-4e1825236d83 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.219849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba202-592a-4e05-b20b-4e1825236d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.250878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:23.261128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cf0ba202-592a-4e05-b20b-4e1825236d83 None None] MFA Rules not processed for user `d54dbe457d64489f9f55abe54bd72bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:23.262332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cf0ba202-592a-4e05-b20b-4e1825236d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.282690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f2739-fd38-4bda-8ae8-2839df8454b6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.328571 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 961/1932] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:23.330186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.330603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.330945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.347385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.348044 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.348315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.348588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] PATH_INFO: `/v3/projects/44f36c0fd60f4665be6d117d4e6c88b4` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.349768 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f36c0fd60f4665be6d117d4e6c88b4)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.351761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.362344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] MFA Rules not processed for user `1e185294e40a49179464b52e30091739`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:23.363515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.398888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f36c0fd60f4665be6d117d4e6c88b4', 'request_id': 'req-22903841-b1c3-4ea6-a08d-695475923b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.406782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f36c0fd60f4665be6d117d4e6c88b4', 'request_id': 'req-22903841-b1c3-4ea6-a08d-695475923b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.420071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f36c0fd60f4665be6d117d4e6c88b4', 'request_id': 'req-22903841-b1c3-4ea6-a08d-695475923b79'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.431140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] The token cache is being invalidate because project 44f36c0fd60f4665be6d117d4e6c88b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:23.431576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.432130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.432440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 962/1933] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:23.433187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.434454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.437558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 972/1934] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:23 2022] DELETE /identity/v3/projects/44f36c0fd60f4665be6d117d4e6c88b4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:23.439409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.442016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.492889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce12c7a-05e6-4b8f-b4d6-873df0fb797f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.505981 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22903841-b1c3-4ea6-a08d-695475923b79 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.510616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.511326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.511606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.511875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.512508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.514491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.536687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:23.542581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:23.554755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 973/1935] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:23.555790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.557617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.566267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.567070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.567390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.567662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.573032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.627626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf9fa1-31d0-44fc-9381-890823aec0a8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.632661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.633383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.633738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.634092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] PATH_INFO: `/v3/users/0df203ee6464493590bda822473b94b6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.639379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df203ee6464493590bda822473b94b6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.639613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.641119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.641810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.651621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 963/1936] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:23.652413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.665362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5068537-2e03-4157-8178-ed6b407b76d2 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.671086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.672067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.672551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.672830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.675498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.677566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.710616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df203ee6464493590bda822473b94b6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.714171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 964/1937] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:23 2022] POST /identity/v3/users => generated 298 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:23.724346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.741110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df203ee6464493590bda822473b94b6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.749711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df203ee6464493590bda822473b94b6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.749833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f36b23-e0d2-4cdb-82e7-54652f72ff8a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.750796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df203ee6464493590bda822473b94b6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.751644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df203ee6464493590bda822473b94b6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.755370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.756111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.756414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.756801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.757438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.759126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.767997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 974/1938] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:23 2022] DELETE /identity/v3/users/0df203ee6464493590bda822473b94b6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:23.768886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.770269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.773568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 965/1939] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:23.780200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.790621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:23.815440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccecbb-8746-4f27-a1f2-d4d139b8c92e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.851470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70788d21-8c6a-4819-adde-f117a0ed649f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:23.857432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.858137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.858487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.858783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] PATH_INFO: `/v3/users/c29f461191ed49fa84d745513048888a` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.860621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29f461191ed49fa84d745513048888a)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.862085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.888678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:23.889626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:23.889869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:23.890141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] PATH_INFO: `/v3/projects/8f565417d38345829f53a2ddee402224/users/b5575fd030d24a99a9341f898f6e8bc0/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:23.907177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:23.922530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f565417d38345829f53a2ddee402224, user_id=b5575fd030d24a99a9341f898f6e8bc0, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:23.927358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:23.947525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29f461191ed49fa84d745513048888a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.948410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 966/1940] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:23 2022] PUT /identity/v3/projects/8f565417d38345829f53a2ddee402224/users/b5575fd030d24a99a9341f898f6e8bc0/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:23.952265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:23.963116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29f461191ed49fa84d745513048888a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.972675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29f461191ed49fa84d745513048888a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.973770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29f461191ed49fa84d745513048888a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.974083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-4830394b-de6d-4b64-a489-d295e76fd62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29f461191ed49fa84d745513048888a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:23.995671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 975/1941] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:23 2022] DELETE /identity/v3/users/c29f461191ed49fa84d745513048888a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:24.001714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.002010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.002284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.032795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:24.034531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.046446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:24.087011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2dcec-6612-4839-8bdb-b3be1cc02a31 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.151967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 976/1942] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:24.158993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.160563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.182356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.183203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.183541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.183877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.195533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.197892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.229305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 967/1943] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:24.229305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.229707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.245350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96d03-fcbf-437d-bffb-cce62943ad29 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.251806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.252567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.253052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.253270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] PATH_INFO: `/v3/projects/ddea359807ad4437a531b7a08a7dd20d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.254547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddea359807ad4437a531b7a08a7dd20d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.256581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.262246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71a171-40cd-4677-8ecb-c362552152bc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.268319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.269140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.269426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.269701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.295190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.295190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.309009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 977/1944] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:24.309823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.312765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.349964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a5cb4-81e6-460a-8694-496217d05614 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.366212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddea359807ad4437a531b7a08a7dd20d', 'request_id': 'req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.379495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddea359807ad4437a531b7a08a7dd20d', 'request_id': 'req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.391662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddea359807ad4437a531b7a08a7dd20d', 'request_id': 'req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.419838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] The token cache is being invalidate because project ddea359807ad4437a531b7a08a7dd20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:24.420294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.423314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b50b9ddd-c2ba-4266-b74c-f10525ee1ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.430385 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 968/1945] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:24 2022] DELETE /identity/v3/projects/ddea359807ad4437a531b7a08a7dd20d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:24.432819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c51443-864b-4594-bf0b-e611e649e110 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.435686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c51443-864b-4594-bf0b-e611e649e110 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.436188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c51443-864b-4594-bf0b-e611e649e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.436726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:24.466101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.466924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.467298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.467784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] PATH_INFO: `/v3/projects/8f565417d38345829f53a2ddee402224/users/b5575fd030d24a99a9341f898f6e8bc0/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.470136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f565417d38345829f53a2ddee402224, user_id=b5575fd030d24a99a9341f898f6e8bc0, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.473158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b425e-d690-4a7d-84d6-b5d9e0e93a32 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.486308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 978/1946] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:24 2022] PUT /identity/v3/projects/8f565417d38345829f53a2ddee402224/users/b5575fd030d24a99a9341f898f6e8bc0/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:24.497931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a8c51443-864b-4594-bf0b-e611e649e110 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:24.499047 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.499447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.499634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8c51443-864b-4594-bf0b-e611e649e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.499762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.556117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] MFA Rules not processed for user `b5575fd030d24a99a9341f898f6e8bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:24.557795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.622428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 969/1947] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:24.627975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.628470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.628887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.655736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:24.657189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.674357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 970/1948] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:24.678821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 979/1949] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:24.684812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.698745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.700576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:24.701054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.720906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba29bd-847f-4800-be5e-7fe6b4b39aa6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.730750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.731394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.731680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.731953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.732776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:24.733140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.745473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.769686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 971/1950] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:24.771688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624c707-0e29-4562-858e-d4bf6cf9ea35 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.774015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.776159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.776849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.777182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.777511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.779519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.787407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1d855-0820-49eb-aaad-ce1952239e99 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.794541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.795467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.795757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.796020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.797364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.798740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.836377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 972/1951] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:24.837808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.840008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:24.855699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.857588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.867038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 980/1952] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:24.868028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.879140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ede2fb-dbf4-42eb-8df0-6158a1344cb6 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.890039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:24.919584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47ec-d2e4-4bac-a731-a670b0f93c84 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:24.924494 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.925137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.925412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.925681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] PATH_INFO: `/v3/projects/958a794c415743f2b259ae93b41e3e4a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.926800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958a794c415743f2b259ae93b41e3e4a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.929103 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.953347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:24.954208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:24.954663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:24.954997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:24.955797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:24.958293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:24.989710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958a794c415743f2b259ae93b41e3e4a', 'request_id': 'req-9592e873-73d0-45ec-8a7f-713d13855498'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:24.995233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 981/1953] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:24 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:24.997274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:24.998544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958a794c415743f2b259ae93b41e3e4a', 'request_id': 'req-9592e873-73d0-45ec-8a7f-713d13855498'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.008358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87f4d-6190-4272-8a0a-e40ff837ef30 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.009245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958a794c415743f2b259ae93b41e3e4a', 'request_id': 'req-9592e873-73d0-45ec-8a7f-713d13855498'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.020943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] The token cache is being invalidate because project 958a794c415743f2b259ae93b41e3e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:25.022743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.023543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.028703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 973/1954] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:24 2022] DELETE /identity/v3/projects/958a794c415743f2b259ae93b41e3e4a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:25.030300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.031474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.095785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.096500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.096897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.097074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.097737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.099850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.109445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592e873-73d0-45ec-8a7f-713d13855498 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.114851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 982/1955] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:25.114994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.114994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.115232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.115336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] PATH_INFO: `/v3/users/07a36d347af74735b8f729ba0675444a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.115595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.117417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a36d347af74735b8f729ba0675444a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.118487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.118599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.120811 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:25.138679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90cd9d-5895-4b61-9964-2d5b955d732a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.144002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.144638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.144933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.145179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.145823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.147454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.160028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 983/1956] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:25.165342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.186866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a36d347af74735b8f729ba0675444a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.188349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8f179-4ed5-47f3-9e0e-1870907c17aa admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.193878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.194860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.195264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.195632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] PATH_INFO: `/v3/projects/a8ad61fc6368473aac3c36820fd69b36/users/f352458b66a44a9b945808c3e18c3f3d/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.200647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a36d347af74735b8f729ba0675444a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.201448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a36d347af74735b8f729ba0675444a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.213809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a36d347af74735b8f729ba0675444a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.214590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a36d347af74735b8f729ba0675444a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.217144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ad61fc6368473aac3c36820fd69b36, user_id=f352458b66a44a9b945808c3e18c3f3d, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.220078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27b2e7-30a6-4f87-a1bf-84e38099c999 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.237126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 984/1957] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:25 2022] PUT /identity/v3/projects/a8ad61fc6368473aac3c36820fd69b36/users/f352458b66a44a9b945808c3e18c3f3d/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:25.238001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 974/1958] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:25 2022] DELETE /identity/v3/users/07a36d347af74735b8f729ba0675444a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:25.247466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.247773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.248036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.295592 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] MFA Rules not processed for user `f352458b66a44a9b945808c3e18c3f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:25.299318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.358387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:25.370807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 985/1959] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:25.375694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.377725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.451024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0b12c-8bc0-4762-880c-a92e1f296b9c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.455995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.456615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.470864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.470864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.470864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.470864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.494783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 975/1960] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:25 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:25.496078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.507281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c8e2-ec63-4a41-8a4c-0e084563d59b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.549827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.552644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.573804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.574587 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.574868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.575140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.576036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.578333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.608682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 986/1961] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:25 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:25.609673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.627354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcd20f-6c76-49b4-a8e0-d755b61b827d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.632982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.633633 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.633911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.634182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.637092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.646347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.664687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 987/1962] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:25.666210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.677005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24a396-d872-425a-883c-524a8b4384c8 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.679416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d36c9-bbe7-4b91-98d8-45d0ab4c3b91 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.682505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.683349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.687183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.687183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] PATH_INFO: `/v3/projects/dbcac58c5b844b559487362635179edd/users/bb79efc6a71a49dba30904116b7ff50b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.692675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.693354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.693650 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.693924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] PATH_INFO: `/v3/projects/4ccd62e38ece475ca70e4d80b0b65328` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.699718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccd62e38ece475ca70e4d80b0b65328)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.704197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcac58c5b844b559487362635179edd, user_id=bb79efc6a71a49dba30904116b7ff50b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.706921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.707689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.729697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:25.739779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 988/1963] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:25 2022] PUT /identity/v3/projects/dbcac58c5b844b559487362635179edd/users/bb79efc6a71a49dba30904116b7ff50b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:25.743958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.793591 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccd62e38ece475ca70e4d80b0b65328', 'request_id': 'req-d9374643-c686-4b30-a875-0d2ad098b97b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.803085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccd62e38ece475ca70e4d80b0b65328', 'request_id': 'req-d9374643-c686-4b30-a875-0d2ad098b97b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.820213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccd62e38ece475ca70e4d80b0b65328', 'request_id': 'req-d9374643-c686-4b30-a875-0d2ad098b97b'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.828718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8c4a8-96c5-47e7-a8ed-eb867c0f34eb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.850981 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] The token cache is being invalidate because project 4ccd62e38ece475ca70e4d80b0b65328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:25.851311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.852087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:25.853764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 976/1964] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:25 2022] DELETE /identity/v3/projects/4ccd62e38ece475ca70e4d80b0b65328 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:25.857726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.860053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.891478 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:25.906572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.907414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.907760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.908094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.908817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.910529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.931974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 989/1965] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:25.937150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:25.939065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:25.953134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee1cdb-3c6f-47dd-aa0f-45b19dbdff86 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.957567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9374643-c686-4b30-a875-0d2ad098b97b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:25.963278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.963278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.963278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.963278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] PATH_INFO: `/v3/projects/dbcac58c5b844b559487362635179edd/users/bb79efc6a71a49dba30904116b7ff50b/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.963611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:25.963611 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.963821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcac58c5b844b559487362635179edd, user_id=bb79efc6a71a49dba30904116b7ff50b, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.963908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.963908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] PATH_INFO: `/v3/users/78966b244dd242fe931653deab0ee326` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:25.964836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78966b244dd242fe931653deab0ee326)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:25.966163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10bb1a-c2b5-4485-8fee-8924654ae201 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.966758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:25.979056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 990/1966] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:25 2022] PUT /identity/v3/projects/dbcac58c5b844b559487362635179edd/users/bb79efc6a71a49dba30904116b7ff50b/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:25.990344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:25.990626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:25.990897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:26.021948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78966b244dd242fe931653deab0ee326'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.023219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] MFA Rules not processed for user `bb79efc6a71a49dba30904116b7ff50b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:26.024580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.034409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78966b244dd242fe931653deab0ee326'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.034857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78966b244dd242fe931653deab0ee326'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.042087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78966b244dd242fe931653deab0ee326'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.042425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78966b244dd242fe931653deab0ee326'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.057165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 977/1967] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:25 2022] DELETE /identity/v3/users/78966b244dd242fe931653deab0ee326 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:26.105621 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 991/1968] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:26.117068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:26.118670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.135261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:26.188046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7a1df-b9a8-464c-9101-7fd126d3036a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:26.192462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:26.193099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:26.193373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:26.193639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:26.262540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:26.264130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:26.273887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 978/1969] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:26.359714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:26.361199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.376637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:26.378797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.438121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b487fa1-88d1-47a9-9be8-6ca6d2b26e75 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:26.442956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:26.448112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:26.448405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:26.448724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:26.455937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.464952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83d329-312c-45c8-927a-5011e8c6adcf service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:26.469605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:26.470246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:26.470649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:26.470777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] PATH_INFO: `/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:26.474362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5c7f6f7b604e5fa899430930ac17ff)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:26.476311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:26.540569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:26.541559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5c7f6f7b604e5fa899430930ac17ff', 'request_id': 'req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.543164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:26.552787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5c7f6f7b604e5fa899430930ac17ff', 'request_id': 'req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.558741 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 992/1970] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:26.571400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5c7f6f7b604e5fa899430930ac17ff', 'request_id': 'req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.577476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:26.587117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] The token cache is being invalidate because project 4c5c7f6f7b604e5fa899430930ac17ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:26.587888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.588698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:26.590474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 979/1971] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:26 2022] DELETE /identity/v3/projects/4c5c7f6f7b604e5fa899430930ac17ff => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:26.617143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:26.627847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a579978-8b17-46e0-bcf0-34bcc699cef4 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:26.645880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:26.696013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:26.696941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:26.697419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:26.697828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:26.699977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:26.786180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:26.788094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:26.802644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 993/1972] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:27.558430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:27.560124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:27.579963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e7839-f8b4-489e-bb3d-f0ecae9499c1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:27.588124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:27.588970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:27.589464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:27.590149 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:27.592553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:27.712878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:27.717158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce45cb5-6b4a-4668-af4c-a9dad8e7fcfa service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:27.730484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 980/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:30.896815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:30.898735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:30.971632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061c910-6dc1-4e12-a54d-eb8d9f381088 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:30.976897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:30.977617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:30.977931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:30.978258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] PATH_INFO: `/v3/users/f352458b66a44a9b945808c3e18c3f3d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:30.979455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f352458b66a44a9b945808c3e18c3f3d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:30.981409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:31.034197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f352458b66a44a9b945808c3e18c3f3d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.050032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f352458b66a44a9b945808c3e18c3f3d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.056988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f352458b66a44a9b945808c3e18c3f3d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.057733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f352458b66a44a9b945808c3e18c3f3d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.058335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f352458b66a44a9b945808c3e18c3f3d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.061374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:31.074539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 994/1974] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:30 2022] DELETE /identity/v3/users/f352458b66a44a9b945808c3e18c3f3d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:31.078867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:31.079347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:31.079463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:31.110762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:31.112032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:31.181750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 981/1975] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:31.192279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:31.194065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:31.266267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c2e66-82d8-4188-8b0b-5db13cb43761 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:31.270243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:31.270976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:31.271302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:31.271599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:31.281661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:31.283132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:31.293856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 995/1976] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:31.707880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:31.709342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:31.731872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a1f0d-383c-4417-9b64-a93929ae1632 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:31.743569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:31.744495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:31.744924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:31.745382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] PATH_INFO: `/v3/users/0205327d17d04f15955792630eac762a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:31.746792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0205327d17d04f15955792630eac762a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:31.748962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:31.802527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:31.804123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:31.820287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7ca1-aaaa-49a4-a4f4-438f42c7fad2 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:31.828366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:31.829018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:31.829296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:31.829568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] PATH_INFO: `/v3/projects/a8ad61fc6368473aac3c36820fd69b36` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:31.833319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0205327d17d04f15955792630eac762a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.834706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ad61fc6368473aac3c36820fd69b36)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:31.837599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:31.878602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0205327d17d04f15955792630eac762a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.879560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0205327d17d04f15955792630eac762a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.887662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0205327d17d04f15955792630eac762a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.888374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-0f6824a8-e468-40ae-a4e8-e7622b94812f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0205327d17d04f15955792630eac762a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.903812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:31.912398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 982/1977] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:31 2022] DELETE /identity/v3/users/0205327d17d04f15955792630eac762a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:31.931198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:31.933829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:31.934395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:31.958073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ad61fc6368473aac3c36820fd69b36', 'request_id': 'req-ddaaab67-1617-4655-850d-08a25a6a4d03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.966834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ad61fc6368473aac3c36820fd69b36', 'request_id': 'req-ddaaab67-1617-4655-850d-08a25a6a4d03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:31.970517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:31.972008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:31.995215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ad61fc6368473aac3c36820fd69b36', 'request_id': 'req-ddaaab67-1617-4655-850d-08a25a6a4d03'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.024238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] The token cache is being invalidate because project a8ad61fc6368473aac3c36820fd69b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:32.024918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.026262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.029327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 996/1978] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:31 2022] DELETE /identity/v3/projects/a8ad61fc6368473aac3c36820fd69b36 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:32.033780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:32.037674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:32.081236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 983/1979] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:32.095737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:32.097295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:32.139387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaab67-1617-4655-850d-08a25a6a4d03 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:32.145451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:32.146672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:32.147018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:32.147572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] PATH_INFO: `/v3/users/bb79efc6a71a49dba30904116b7ff50b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:32.149358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb79efc6a71a49dba30904116b7ff50b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:32.151921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:32.189817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:32.195537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4ab8e-fee5-4d1d-ab19-fc489132960a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:32.201171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:32.202182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:32.202454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:32.202876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:32.228719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:32.230414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:32.240232 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb79efc6a71a49dba30904116b7ff50b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.241538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 984/1980] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:32.253372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb79efc6a71a49dba30904116b7ff50b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.261094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb79efc6a71a49dba30904116b7ff50b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.263235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb79efc6a71a49dba30904116b7ff50b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.263235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb79efc6a71a49dba30904116b7ff50b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:32.279131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 997/1981] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:32 2022] DELETE /identity/v3/users/bb79efc6a71a49dba30904116b7ff50b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:32.737473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:32.752891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08ab4e-8252-43f6-a409-e9e71cded1a6 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:32.790858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:32.793040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:32.856271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:32.857215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:32.868162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:32.868543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] PATH_INFO: `/v3/projects/dbcac58c5b844b559487362635179edd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:32.872338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbcac58c5b844b559487362635179edd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:32.878886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:32.956206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631495ce-4b8c-43dc-95ad-9c632e75d18f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:32.964631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:32.965540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:32.966067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:32.968987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] PATH_INFO: `/v3/projects/c7dd2666ad8c417b9a76b0ec4112d908` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:32.979708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7dd2666ad8c417b9a76b0ec4112d908)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:32.986884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.015075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:33.023906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbcac58c5b844b559487362635179edd', 'request_id': 'req-ebed87fc-54fe-4736-bdcd-564256538f83'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.048678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbcac58c5b844b559487362635179edd', 'request_id': 'req-ebed87fc-54fe-4736-bdcd-564256538f83'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.064884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbcac58c5b844b559487362635179edd', 'request_id': 'req-ebed87fc-54fe-4736-bdcd-564256538f83'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.101062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] The token cache is being invalidate because project dbcac58c5b844b559487362635179edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:33.102791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.103747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ebed87fc-54fe-4736-bdcd-564256538f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.106082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 985/1982] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:32 2022] DELETE /identity/v3/projects/dbcac58c5b844b559487362635179edd => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:33.112036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a882d-6cd4-41e1-9bd2-ae051806d395 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.112672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a882d-6cd4-41e1-9bd2-ae051806d395 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.113130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a882d-6cd4-41e1-9bd2-ae051806d395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.129945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7dd2666ad8c417b9a76b0ec4112d908', 'request_id': 'req-38736efc-4954-47d2-8dee-32ffeeb11a2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.137865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7dd2666ad8c417b9a76b0ec4112d908', 'request_id': 'req-38736efc-4954-47d2-8dee-32ffeeb11a2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.149502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e09a882d-6cd4-41e1-9bd2-ae051806d395 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:33.151550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e09a882d-6cd4-41e1-9bd2-ae051806d395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:33.157417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7dd2666ad8c417b9a76b0ec4112d908', 'request_id': 'req-38736efc-4954-47d2-8dee-32ffeeb11a2c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.180373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] The token cache is being invalidate because project c7dd2666ad8c417b9a76b0ec4112d908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:33.180609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.182923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.185949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 998/1983] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:32 2022] DELETE /identity/v3/projects/c7dd2666ad8c417b9a76b0ec4112d908 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:33.191581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.195607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:33.283717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:33.313278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 986/1984] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:33.324796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.324796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.324796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.379223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38736efc-4954-47d2-8dee-32ffeeb11a2c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.381896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:33.383204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:33.394683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.395437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.395721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.396028 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] PATH_INFO: `/v3/users/1e185294e40a49179464b52e30091739` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.399066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e185294e40a49179464b52e30091739)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.401570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.408303 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 987/1985] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:33.409560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.425367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088a901-89c0-486e-b30b-7f55281ea60d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.430697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.431457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.431761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.432037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.433446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:33.433788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.435767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.456167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 988/1986] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:33.463793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.478601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7adcf-e162-4114-b1b2-c054318fa687 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.487223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.488528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.488967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.489383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.490207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.494153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.525553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e185294e40a49179464b52e30091739'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.546175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 989/1987] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:33 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:33.551933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.570492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e185294e40a49179464b52e30091739'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.594591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e185294e40a49179464b52e30091739'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.595411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e185294e40a49179464b52e30091739'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.596030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e185294e40a49179464b52e30091739'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:33.633036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 999/1988] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:33 2022] DELETE /identity/v3/users/1e185294e40a49179464b52e30091739 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:33.680273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6ea0f-d03b-4dee-8888-ca808716fb17 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.758272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:33.784051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.785216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.785216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.785444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.786150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.788371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.817941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 990/1989] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:33 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:33.819053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.820863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:33.836072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fff5e-2b98-42b9-9b76-77b866b1d3f3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.842131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.842925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.843484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.843613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.844316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.846609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.863555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1000/1990] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:33.864838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.887296 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8aed0-b74e-4c83-ba86-3fca3c5061c0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.894825 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.895492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.896336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.896927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.898734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.903280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6cfb5-ad40-4bc2-aaf0-08907667b28f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.920537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 991/1991] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:33.923938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:33.939406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c1a90-d88d-4b96-a85a-2cfa5e3b38da admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:33.945837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:33.946627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:33.946950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:33.947312 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] PATH_INFO: `/v3/projects/6ba20be7f6024508ae6751364b56b6c1/users/594034b8d0024b43942b32aeed4106b2/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:33.977158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba20be7f6024508ae6751364b56b6c1, user_id=594034b8d0024b43942b32aeed4106b2, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:33.979934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:33.997287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1001/1992] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:33 2022] PUT /identity/v3/projects/6ba20be7f6024508ae6751364b56b6c1/users/594034b8d0024b43942b32aeed4106b2/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:34.009939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:34.010231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:34.010512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:34.044138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] MFA Rules not processed for user `594034b8d0024b43942b32aeed4106b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:34.045748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:34.073509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:34.075317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:34.182097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4c833-75b7-4041-9852-da77f561ef57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:34.190502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:34.191571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:34.197385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:34.197385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] PATH_INFO: `/v3/projects/bec34418a2a94c278cab0ae915c4ec69` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:34.197385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec34418a2a94c278cab0ae915c4ec69)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:34.197385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:34.208366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 992/1993] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:34.218078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:34.220733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:34.222585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:34.263551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec34418a2a94c278cab0ae915c4ec69', 'request_id': 'req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.273196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec34418a2a94c278cab0ae915c4ec69', 'request_id': 'req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.283939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec34418a2a94c278cab0ae915c4ec69', 'request_id': 'req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.295900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf50409-da29-4d5a-8263-98fd243e4809 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:34.297305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] The token cache is being invalidate because project bec34418a2a94c278cab0ae915c4ec69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:34.298857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.298857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.299977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1002/1994] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:34 2022] DELETE /identity/v3/projects/bec34418a2a94c278cab0ae915c4ec69 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:34.304090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:34.305711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:34.372523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:34.373490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:34.374067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:34.374498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:34.379580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:34.400446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:34.419085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f269d6b-3f43-4873-aa9d-e3e88c91a62e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:34.429456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:34.430122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:34.430421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:34.430679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] PATH_INFO: `/v3/users/b5575fd030d24a99a9341f898f6e8bc0` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:34.432411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5575fd030d24a99a9341f898f6e8bc0)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:34.434803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:34.450898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:34.452622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:34.465049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 993/1995] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:34.494173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5575fd030d24a99a9341f898f6e8bc0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.510683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5575fd030d24a99a9341f898f6e8bc0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.522659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5575fd030d24a99a9341f898f6e8bc0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.523722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5575fd030d24a99a9341f898f6e8bc0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.524394 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f888e0d3-2bee-4f6f-9fa1-ff257a3027cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5575fd030d24a99a9341f898f6e8bc0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:34.547138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1003/1996] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:34 2022] DELETE /identity/v3/users/b5575fd030d24a99a9341f898f6e8bc0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:35.097579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.117482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e882104-3350-4fd4-89b0-a926288ec2ab service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.234793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.235680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.235788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.236021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] PATH_INFO: `/v3/projects/8f565417d38345829f53a2ddee402224` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.237567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f565417d38345829f53a2ddee402224)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.240302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.295515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f565417d38345829f53a2ddee402224', 'request_id': 'req-62325e89-7cb3-4589-bf1f-858824f676e4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.304548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f565417d38345829f53a2ddee402224', 'request_id': 'req-62325e89-7cb3-4589-bf1f-858824f676e4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.317614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f565417d38345829f53a2ddee402224', 'request_id': 'req-62325e89-7cb3-4589-bf1f-858824f676e4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.339265 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] The token cache is being invalidate because project 8f565417d38345829f53a2ddee402224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:35.339593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.340207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62325e89-7cb3-4589-bf1f-858824f676e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.341916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 994/1997] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:35 2022] DELETE /identity/v3/projects/8f565417d38345829f53a2ddee402224 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:35.346819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.347381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.347457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.376268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:35.377997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.404431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:35.467022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1004/1998] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:35.479341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.479341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.479341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.507883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:35.509170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.520099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 995/1999] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:35.524848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.535065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383ec44-e864-44c9-8d0e-6c914939f59f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.540000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.540707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.541046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.541378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.542200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:35.542876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.546761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.547657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.548850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.563310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1005/2000] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:25:35.563538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321afe4-9618-41db-9ac3-81c1dc78ab2f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.565257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.567028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.567772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.567995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.568264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] PATH_INFO: `/v3/users/37a30e3348be4d02b8b827b7eb32ba6a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.569455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a30e3348be4d02b8b827b7eb32ba6a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.571419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.575369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8271ba4-6861-4b94-9977-27cef9e161e1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.580102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.580749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.581031 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.581301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.581963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.584810 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.614737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1006/2001] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:35.614943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:35.619653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.625701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a30e3348be4d02b8b827b7eb32ba6a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.639804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a30e3348be4d02b8b827b7eb32ba6a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.640445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a30e3348be4d02b8b827b7eb32ba6a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.647555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a30e3348be4d02b8b827b7eb32ba6a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.648337 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a30e3348be4d02b8b827b7eb32ba6a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.662795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 996/2002] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:35 2022] DELETE /identity/v3/users/37a30e3348be4d02b8b827b7eb32ba6a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:35.690351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c1075-694d-4bf0-bee5-3b92a01f8956 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.750537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.751241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.751558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.751813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.752447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.760317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.785739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1007/2003] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:35.789760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.791538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.804801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669bdaa-5fe3-43f4-9975-a742b14bc713 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.809560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.810166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.810440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.810702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.811438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.813035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.825635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 997/2004] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:35.827777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.839182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f94a3-e44b-476e-90f3-ba3768111a14 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.844181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.844896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.845178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.845447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.846091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.848242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.850075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.852200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.872159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1008/2005] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:35.872594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834192b-9040-4fbe-8ab9-1bffa7eb74ba admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.872699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:35.873505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.874190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.874466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.874755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] PATH_INFO: `/v3/projects/3c9d4b857e654cb38c82b0a28c5ccd77` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.876128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9d4b857e654cb38c82b0a28c5ccd77)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.878225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.888864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6b58b-c4bf-4e82-bcf9-69633e8c0164 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:35.894156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:35.894828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.895108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.895459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] PATH_INFO: `/v3/projects/aaa45dccffd64489a30da96f93cf802f/users/44a2023bb4dc49838b8d5db6a2f155d9/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.905080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:35.912861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa45dccffd64489a30da96f93cf802f, user_id=44a2023bb4dc49838b8d5db6a2f155d9, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:35.916396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a943b-37b9-47e7-a049-54bb7c0cd5cb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:35.934154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1009/2006] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:35 2022] PUT /identity/v3/projects/aaa45dccffd64489a30da96f93cf802f/users/44a2023bb4dc49838b8d5db6a2f155d9/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:35.937976 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9d4b857e654cb38c82b0a28c5ccd77', 'request_id': 'req-db7c282a-d150-4948-a87f-252fa198a9fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.939313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.939603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.939875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:35.952162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9d4b857e654cb38c82b0a28c5ccd77', 'request_id': 'req-db7c282a-d150-4948-a87f-252fa198a9fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.960411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:35.961670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:35.963282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9d4b857e654cb38c82b0a28c5ccd77', 'request_id': 'req-db7c282a-d150-4948-a87f-252fa198a9fc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.981559 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] The token cache is being invalidate because project 3c9d4b857e654cb38c82b0a28c5ccd77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:35.981906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.982696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-db7c282a-d150-4948-a87f-252fa198a9fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:35.988061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 998/2007] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:35 2022] DELETE /identity/v3/projects/3c9d4b857e654cb38c82b0a28c5ccd77 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:25:35.989322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3da7-0578-4518-a5ce-20fd056e77a9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:35.989603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3da7-0578-4518-a5ce-20fd056e77a9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:35.989925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a3da7-0578-4518-a5ce-20fd056e77a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.016071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-243a3da7-0578-4518-a5ce-20fd056e77a9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:36.017283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-243a3da7-0578-4518-a5ce-20fd056e77a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:36.035330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1010/2008] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:36.046671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.049347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:36.085566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 999/2009] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:36.094806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.095216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.095503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.121440 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:36.123381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:36.123543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e874-c42c-4679-aa7d-62adb3eac73a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.131387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.133076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.133357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.133626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.133966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1000/2010] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:36.138701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.149179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.150958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.153351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177aaae-8abb-44d3-a046-3a7029318cf6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.162916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1011/2011] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:36.165745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.166367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.166696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.166970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.167670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:36.167972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.169586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.187020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1001/2012] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:36.191014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.207482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485305b-3e9a-4e34-9e0c-6dd85c47d02e service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.237106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.239089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.239574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.240002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.240514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.242457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.303302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1012/2013] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:36 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:25:36.307612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.323861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700eece-7890-4405-9190-065b181a6f7f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.331490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:36.394605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.395352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.395640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.395942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.396544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.398460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.423181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1002/2014] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:36 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:36.425508 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.442025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c2ed-3b1e-4958-b3d7-6862fb8ef86f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.447759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.448489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.448706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.449040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.449753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.451568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.462540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1013/2015] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:36.466754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.478130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4bd86-b5ff-4266-acf2-4f61dc17f11d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.485577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.486655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.487468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.487932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.488827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.490863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4496f91-7835-4ca2-af3f-f68f0e1f001d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.520942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1003/2016] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:36.525853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.543757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22ae09-5bd3-437b-89f8-cda93982719e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.545992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.547001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.547001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.547290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] PATH_INFO: `/v3/projects/4029646841bb4bceb1dad1df19c4b870/users/cea2bf367ebd492b8bcbf2ed6b17e747/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.563977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4029646841bb4bceb1dad1df19c4b870, user_id=cea2bf367ebd492b8bcbf2ed6b17e747, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.566777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.584485 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1014/2017] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:36 2022] PUT /identity/v3/projects/4029646841bb4bceb1dad1df19c4b870/users/cea2bf367ebd492b8bcbf2ed6b17e747/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:36.589803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56712-25ba-4b10-8eac-15c26a38d0dd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.590161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56712-25ba-4b10-8eac-15c26a38d0dd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.590502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56712-25ba-4b10-8eac-15c26a38d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.617035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-9ec56712-25ba-4b10-8eac-15c26a38d0dd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:36.618549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9ec56712-25ba-4b10-8eac-15c26a38d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:36.697567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1004/2018] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:36.708205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:36.745637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:36.771703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae346e-8a62-4a7a-b747-dc3f70eb2e95 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:36.775483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:36.776173 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.776519 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.776882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.786649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:36.788204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:36.798775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1015/2019] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:36.945792 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712826-2f3a-4976-a2a7-1582d39cb576 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:36.946261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712826-2f3a-4976-a2a7-1582d39cb576 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:36.946614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51712826-2f3a-4976-a2a7-1582d39cb576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:36.979950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-51712826-2f3a-4976-a2a7-1582d39cb576 None None] MFA Rules not processed for user `44a2023bb4dc49838b8d5db6a2f155d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:36.981356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-51712826-2f3a-4976-a2a7-1582d39cb576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:37.063605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1005/2020] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:37.074913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:37.078377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:37.145770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe42f7-7f41-44cb-9168-514f1bd5f4e1 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:37.149387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:37.150000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:37.150427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:37.150658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:37.163834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:37.165402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914cc69-8b09-44d0-8bc2-3cd772a8640f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:37.175853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1016/2021] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:40.808885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:40.809487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:40.809487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:40.838287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] MFA Rules not processed for user `44a2023bb4dc49838b8d5db6a2f155d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:40.840729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:40.865087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1006/2022] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:41.433828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82501b-8fff-445c-8b6b-7b58a98d175d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:41.434306 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82501b-8fff-445c-8b6b-7b58a98d175d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:41.434709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82501b-8fff-445c-8b6b-7b58a98d175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:41.473632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4d82501b-8fff-445c-8b6b-7b58a98d175d None None] MFA Rules not processed for user `cea2bf367ebd492b8bcbf2ed6b17e747`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:41.475206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4d82501b-8fff-445c-8b6b-7b58a98d175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:41.575566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1017/2023] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:41.706918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:41.708735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:41.721514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:41.794704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb0ee-1767-49a2-8ee9-15e0e9246645 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:41.800725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:41.801352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:41.801633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:41.801904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:41.810802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:41.812654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:41.824965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1007/2024] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:43.458045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366516d9-1391-4e04-b037-ca9f7f9c9721 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:43.458819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366516d9-1391-4e04-b037-ca9f7f9c9721 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:43.459101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366516d9-1391-4e04-b037-ca9f7f9c9721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:43.482432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-366516d9-1391-4e04-b037-ca9f7f9c9721 None None] MFA Rules not processed for user `cea2bf367ebd492b8bcbf2ed6b17e747`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:43.483791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-366516d9-1391-4e04-b037-ca9f7f9c9721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:43.492964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1018/2025] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:43.552352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:43.553963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:43.572603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f4-c71d-4d9d-8d23-452c240486ad service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:43.577634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:43.578366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:43.578708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:43.579067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] PATH_INFO: `/v3/users/594034b8d0024b43942b32aeed4106b2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:43.580945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594034b8d0024b43942b32aeed4106b2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:43.583379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:43.657903 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594034b8d0024b43942b32aeed4106b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:43.676197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594034b8d0024b43942b32aeed4106b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:43.679379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594034b8d0024b43942b32aeed4106b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:43.687583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594034b8d0024b43942b32aeed4106b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:43.688305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594034b8d0024b43942b32aeed4106b2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:43.707943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1008/2026] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:43 2022] DELETE /identity/v3/users/594034b8d0024b43942b32aeed4106b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:43.712033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:43.712511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:43.712831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:43.736875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:43.738420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:43.821283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1019/2027] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:43.861747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:43.871962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:43.953121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39547c9b-d45a-4e0d-a12b-9d31d4b436f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:43.960396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:43.961166 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:43.961832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:43.962583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:43.974248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:43.976531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b99223-dcc0-4b86-a6a4-022efbab5c55 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:44.008419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1009/2028] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:44.636133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:44.644346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:44.657123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:44.663427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469c330-f7d4-4630-862b-52ab2139d3d6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:44.669255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:44.670120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:44.670656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:44.671081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] PATH_INFO: `/v3/projects/6ba20be7f6024508ae6751364b56b6c1` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:44.672445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba20be7f6024508ae6751364b56b6c1)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:44.674407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:44.734204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba20be7f6024508ae6751364b56b6c1', 'request_id': 'req-6e48b662-2ed3-44b3-a33e-30d1816e3cde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:44.742338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba20be7f6024508ae6751364b56b6c1', 'request_id': 'req-6e48b662-2ed3-44b3-a33e-30d1816e3cde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:44.752773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba20be7f6024508ae6751364b56b6c1', 'request_id': 'req-6e48b662-2ed3-44b3-a33e-30d1816e3cde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:44.769221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] The token cache is being invalidate because project 6ba20be7f6024508ae6751364b56b6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:44.769662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:44.770785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6e48b662-2ed3-44b3-a33e-30d1816e3cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:44.773056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1020/2029] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:44 2022] DELETE /identity/v3/projects/6ba20be7f6024508ae6751364b56b6c1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:44.778408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:44.779748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:44.780343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:44.809419 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:44.811031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:44.816562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:44.924617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1010/2030] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:44.926077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:44.926509 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:44.926931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:44.975752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:44.979580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:44.989790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1021/2031] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:44.994140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.005683 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5155429-20d9-4143-bced-47df3fa42cdf None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.016011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.017029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.017522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.018404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.020167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:45.021120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.025045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.039140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1011/2032] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:45.043455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.056307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07f8f9-ebb8-48c5-ae0b-43e1bfd8708a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.067262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.071641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.072123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.072531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.073342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.090258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.147647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1022/2033] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:45 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:45.151533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.171657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6003a35-1a5e-4acd-8c72-f7d5db3623e2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.257617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.258416 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.258684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.259995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.259995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.262632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.303644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1012/2034] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:45 2022] POST /identity/v3/users => generated 281 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:45.313263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.332698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a837c-cc26-4e49-bd3f-b95a51119180 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.340661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.341631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.342094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.342571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.343805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.345630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.365345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1023/2035] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:45.370950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.388392 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf6f5d-a4cf-4d52-bc1e-e58cc233d8f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.400720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.402094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.402492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.403415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.404357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.407570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6923c-1ec6-4dd6-87bc-1a53acd3f823 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.433498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1013/2036] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:25:45.435009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.449144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166ac75-67f0-4477-bf95-506ecc018a03 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.466311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.468246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.471865 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.472171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] PATH_INFO: `/v3/projects/a6315aac28f94f94b96964a4fa2534be/users/578fdc8e125842c99fdadc561faf35b1/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.519375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6315aac28f94f94b96964a4fa2534be, user_id=578fdc8e125842c99fdadc561faf35b1, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.522736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.560038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1024/2037] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:45 2022] PUT /identity/v3/projects/a6315aac28f94f94b96964a4fa2534be/users/578fdc8e125842c99fdadc561faf35b1/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:45.568318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.568605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.568880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.616834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] MFA Rules not processed for user `578fdc8e125842c99fdadc561faf35b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:45.618884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:45.761665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1014/2038] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:45.799660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.801402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:45.887051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d16452-df6c-43e3-8994-4309ed75bc5d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:45.906025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:45.906669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:45.906945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:45.907246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:45.923838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:45.925785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f7d96-29d2-4670-9fbe-13f379a54192 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:45.938595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1025/2039] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:45.956127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:45.963669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:46.053268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137b23f-365c-4008-975c-608ae9b48937 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:46.058207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:46.058865 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:46.059159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:46.059445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] PATH_INFO: `/v3/users/578fdc8e125842c99fdadc561faf35b1` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:46.060731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=578fdc8e125842c99fdadc561faf35b1)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:46.062695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:46.127324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578fdc8e125842c99fdadc561faf35b1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.147418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578fdc8e125842c99fdadc561faf35b1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.148243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578fdc8e125842c99fdadc561faf35b1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.157633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578fdc8e125842c99fdadc561faf35b1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.158530 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578fdc8e125842c99fdadc561faf35b1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.178554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1015/2040] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:45 2022] DELETE /identity/v3/users/578fdc8e125842c99fdadc561faf35b1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:46.182985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:46.183493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:46.183912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:46.235788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:46.237813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:46.284785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:46.351756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1026/2041] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:46.361689 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:46.363225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:46.433156 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698e12-2bbc-4bfe-9d28-fd163be6a86f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:46.437783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:46.439118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:46.439529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:46.440008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:46.451634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:46.453236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa103cc-880f-4645-9694-20f70d651012 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:46.475666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1016/2042] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:46.825832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:46.827643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:46.843457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc7145-aecb-4977-940f-d503d2b6f3ca None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:46.849434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:46.850455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:46.850942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:46.851449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] PATH_INFO: `/v3/projects/a6315aac28f94f94b96964a4fa2534be` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:46.853252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6315aac28f94f94b96964a4fa2534be)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:46.855403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:46.929101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6315aac28f94f94b96964a4fa2534be', 'request_id': 'req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.939174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6315aac28f94f94b96964a4fa2534be', 'request_id': 'req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.951007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6315aac28f94f94b96964a4fa2534be', 'request_id': 'req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.963637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] The token cache is being invalidate because project a6315aac28f94f94b96964a4fa2534be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:46.963978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.964658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-f4aefbb8-1c76-492b-9ff4-1ea6cf08fb4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:46.966377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1027/2043] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:46 2022] DELETE /identity/v3/projects/a6315aac28f94f94b96964a4fa2534be => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:46.973297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:46.973829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:46.973829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:46.993709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:46.994921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:47.019364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:47.095373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1017/2044] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:47.108950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.109278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.109516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.145423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:47.146613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:47.155824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1028/2045] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:47.160615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.169935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddc1ba-141d-4da3-b45c-558ad8ce2790 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.174713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.175362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.175640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.175909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.176600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:47.176860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.178475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.194576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1018/2046] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:47.195625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.206459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75bb2-e46e-4d93-8221-eefda54f956a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.212479 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.213179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.213528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.214033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.214768 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.216841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.270035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.271536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1029/2047] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:47 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:47.293568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487021c-b991-4bff-8605-3a96cdc6da5e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.365916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.366570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.366906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.367206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.367861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.370193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.425899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1019/2048] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:47 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:47.426765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.445803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7f87-0116-4aa0-8698-837bfcacb84f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.458399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.459470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.459893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.461029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.462321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.464462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.479619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1030/2049] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:47.481887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.494145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afdfd-0542-4677-a1b9-42d7eabfcb24 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.499876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.501194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.501489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.501760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.502431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.504099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2686818-16a9-4329-b067-4fb6d8fb5ef7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.516737 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1020/2050] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:47.520924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.531526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedfacf-9177-4437-a9f0-f3cf9ccba974 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.536180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.536844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.537126 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.537396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] PATH_INFO: `/v3/projects/4bc21dd502ee4d349f1e0f0ce3a9ed3b/users/03a29b46be0145418eacb6d615f8499a/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.553735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc21dd502ee4d349f1e0f0ce3a9ed3b, user_id=03a29b46be0145418eacb6d615f8499a, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.556182 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.569737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1031/2051] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:47 2022] PUT /identity/v3/projects/4bc21dd502ee4d349f1e0f0ce3a9ed3b/users/03a29b46be0145418eacb6d615f8499a/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:47.574800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b90c8-7c63-4052-9c45-3cca1ad966a5 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.575084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b90c8-7c63-4052-9c45-3cca1ad966a5 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.575400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b90c8-7c63-4052-9c45-3cca1ad966a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.595327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-871b90c8-7c63-4052-9c45-3cca1ad966a5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:47.596437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-871b90c8-7c63-4052-9c45-3cca1ad966a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:47.659357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1021/2052] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:47.669489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:47.671480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:47.729945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:47.743916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799b3c2-6834-4a3e-9f90-587c8fe02748 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:47.748695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:47.749315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:47.749598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:47.749879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:47.759638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:47.761094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:47.771192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1032/2053] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:25:52.453749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.454214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.454389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.484396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] MFA Rules not processed for user `03a29b46be0145418eacb6d615f8499a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:52.485596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:52.506365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:52.576581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1022/2054] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:52.581269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.598135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e24f-7c45-4344-9d73-fd9902a86533 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.606753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.607425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.607702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.608037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.608682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.610942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:52.648020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1033/2055] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:52 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:52.649240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.661421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626aa99-758b-4213-9b83-febbcbd3e886 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.727657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.728349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.728834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.729109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.729805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.731978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:52.755271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1023/2056] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:52 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:52.759390 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.770287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e049d-f403-47c0-adbc-851e5b427507 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.775134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.775763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.776159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.776300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.776958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.778571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:52.790608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1034/2057] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:52.794988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.805539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68279698-1bf6-4970-956f-c6d1d1a4da76 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.810763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.811631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.811912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.812183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] PATH_INFO: `/v3/projects/3063b248020e46bbaaca247b1387027d/users/d359a6c2303d4bf7a91fe7ace3a43f48/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.828625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3063b248020e46bbaaca247b1387027d, user_id=d359a6c2303d4bf7a91fe7ace3a43f48, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.831618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:52.845791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1024/2058] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:52 2022] PUT /identity/v3/projects/3063b248020e46bbaaca247b1387027d/users/d359a6c2303d4bf7a91fe7ace3a43f48/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:52.849842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.860352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a83a-8a5c-4186-b386-d0b6f05dadb6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.920609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.921248 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.921529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.921821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.922500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.924189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:52.938714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1035/2059] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:52.939753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:52.968443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5175001-ad98-4ba4-9b3a-8e7b02c5074d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:52.978736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:52.979129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:52.979454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:52.979773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] PATH_INFO: `/v3/projects/3063b248020e46bbaaca247b1387027d/users/d359a6c2303d4bf7a91fe7ace3a43f48/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:52.987283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3063b248020e46bbaaca247b1387027d, user_id=d359a6c2303d4bf7a91fe7ace3a43f48, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:52.990225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e5f8-afe2-4989-a10d-433d49ec9cdc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:53.037269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1025/2060] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:52 2022] PUT /identity/v3/projects/3063b248020e46bbaaca247b1387027d/users/d359a6c2303d4bf7a91fe7ace3a43f48/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:57.413382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:57.415534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:57.441072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28617-e97e-47c4-9c58-5ce364fa7f4c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:57.581763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:57.582996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:57.583336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:57.583613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] PATH_INFO: `/v3/users/d54dbe457d64489f9f55abe54bd72bec` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:57.595231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54dbe457d64489f9f55abe54bd72bec)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:57.595231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:57.678906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54dbe457d64489f9f55abe54bd72bec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:57.686194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:57.700760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54dbe457d64489f9f55abe54bd72bec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:57.716916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54dbe457d64489f9f55abe54bd72bec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:57.717914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54dbe457d64489f9f55abe54bd72bec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:57.720019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54dbe457d64489f9f55abe54bd72bec'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:57.742694 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1036/2061] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:25:57 2022] DELETE /identity/v3/users/d54dbe457d64489f9f55abe54bd72bec => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:57.744509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:57.745187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:57.745304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:57.781009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:57.782246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:57.885244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1026/2062] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:57.904860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:57.906331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:57.988580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfdfca6-15f0-4693-82c9-5d6cf6989e41 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:57.998776 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:57.999696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:58.000000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:58.000317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:58.020885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:58.023190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf96f9-1c86-4e69-9eee-bd249c8f0964 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:58.046997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1037/2063] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:25:58.832105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:58.835069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:58.861406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d473cc2-6b8d-4829-9a26-340fda32aa8b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:58.888432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:58.889443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:58.889721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:58.889993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] PATH_INFO: `/v3/projects/9f27a977e86a41f3be77b9acf324ef99` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:58.895055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f27a977e86a41f3be77b9acf324ef99)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:58.897503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:58.910345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:58.957206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f27a977e86a41f3be77b9acf324ef99', 'request_id': 'req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:58.965468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f27a977e86a41f3be77b9acf324ef99', 'request_id': 'req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:58.981112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f27a977e86a41f3be77b9acf324ef99', 'request_id': 'req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:58.996036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] The token cache is being invalidate because project 9f27a977e86a41f3be77b9acf324ef99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:25:58.996479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:58.997152 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-efe8a9f0-2d19-4b0c-a9ac-bb50fa9abdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:25:58.999185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1027/2064] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:25:58 2022] DELETE /identity/v3/projects/9f27a977e86a41f3be77b9acf324ef99 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:59.004171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.004687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.004687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.034125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:59.035902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.135411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1038/2065] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:59.143646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.143974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.144276 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.171554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:59.173251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.183050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1028/2066] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:59.187070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.204088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16799d47-40bf-416f-a8a1-49e39f6f82d3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.212058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.212774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.213110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.213385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.214145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:25:59.214536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.217735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.250482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1039/2067] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:25:59.254539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.265928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c537904-3371-41e9-a1f5-743a80acb2cd None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.271649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.272291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.272568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.272864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.273483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.275502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.320412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1029/2068] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:59.324663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.346069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f712fd-763b-4a45-80a8-f468c2ed4e57 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.433850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.434899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.435410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.436562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.437762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.440315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.450010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:59.474241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1040/2069] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:25:59.477906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.490190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7ed6-7891-4458-be6c-cf158cec0a08 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.495044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.495705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.495978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.496243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.496895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.498532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.512082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1030/2070] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:59.513473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.524901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33a9f3-8cec-46da-a8ac-c51eb59306c9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.531843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.532468 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.532818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.533089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.533954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.535656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ac944-0af3-4122-8cea-2d070fb06118 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.570870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1041/2071] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:25:59.570961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.583805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d62b-e1a4-4890-908f-77b908c0eb2b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.595790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.596634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.597866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.598221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] PATH_INFO: `/v3/projects/74a1bdbf22c64613aa72ee2e046adae9/users/27b2ed46bee948a68202e0d6b9b5bdcd/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.638570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a1bdbf22c64613aa72ee2e046adae9, user_id=27b2ed46bee948a68202e0d6b9b5bdcd, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:25:59.649908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.650134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccde4de-0994-43d7-a1a8-ffa8b5f57a7f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:25:59.650222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.650454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.679228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1031/2072] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:25:59 2022] PUT /identity/v3/projects/74a1bdbf22c64613aa72ee2e046adae9/users/27b2ed46bee948a68202e0d6b9b5bdcd/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:25:59.695013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.695271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.695526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.697511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] MFA Rules not processed for user `d359a6c2303d4bf7a91fe7ace3a43f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:59.698841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.754633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] MFA Rules not processed for user `27b2ed46bee948a68202e0d6b9b5bdcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:25:59.756011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.791304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1042/2073] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:25:59.804696 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.813407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.901344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1032/2074] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:25:59.914332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:25:59.917144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:25:59.937446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d43270-ad00-4245-b794-52c495ccd1c4 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:25:59.956124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:25:59.960932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:25:59.961565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:25:59.961851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:25:59.962130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:25:59.964585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:00.004815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1a1a8-4d7d-423e-bd0b-20532046daac None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:00.011797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:00.012435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:00.012710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:00.012998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:00.025275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:00.027324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5843-2cf3-40e0-959b-d5dd8f175d5d service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:00.039242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1033/2075] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:26:00.045903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:00.047557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d725c5-0132-450b-96c3-8174f66d91ce service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:00.056123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:00.073675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1043/2076] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:26:01.343382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:01.343945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:01.343945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:01.363931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] MFA Rules not processed for user `03a29b46be0145418eacb6d615f8499a`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:01.365226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:01.372413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1034/2077] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:02.597503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709754-e7e4-4860-8728-d18f6cd7a0f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:02.598057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709754-e7e4-4860-8728-d18f6cd7a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:02.598057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709754-e7e4-4860-8728-d18f6cd7a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:02.631698 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4e709754-e7e4-4860-8728-d18f6cd7a0f1 None None] MFA Rules not processed for user `27b2ed46bee948a68202e0d6b9b5bdcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:02.633157 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4e709754-e7e4-4860-8728-d18f6cd7a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:02.641113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1044/2078] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:07.492089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:07.494332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:07.573668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48f391-e751-4a6e-b751-d44573015449 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:07.579574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:07.580327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:07.580665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:07.580956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] PATH_INFO: `/v3/users/335ed1e0f6dc4c9f844c5cebdcbcc5dd` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:07.582219 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335ed1e0f6dc4c9f844c5cebdcbcc5dd)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:07.584603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:07.641713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335ed1e0f6dc4c9f844c5cebdcbcc5dd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:07.655424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335ed1e0f6dc4c9f844c5cebdcbcc5dd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:07.656142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335ed1e0f6dc4c9f844c5cebdcbcc5dd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:07.663173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335ed1e0f6dc4c9f844c5cebdcbcc5dd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:07.663829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335ed1e0f6dc4c9f844c5cebdcbcc5dd'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:07.677495 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1035/2079] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:26:07 2022] DELETE /identity/v3/users/335ed1e0f6dc4c9f844c5cebdcbcc5dd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:26:07.682078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873364-2b44-4799-b860-753cd60e008c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:07.682605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873364-2b44-4799-b860-753cd60e008c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:07.682605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a873364-2b44-4799-b860-753cd60e008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:07.704158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2a873364-2b44-4799-b860-753cd60e008c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:07.705393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a873364-2b44-4799-b860-753cd60e008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:07.776423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1045/2080] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:07.787445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:07.788875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:07.835699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:07.854994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ca100-643c-45ca-8b17-4ea7b16888f9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:07.860032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:07.860656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:07.860930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:07.861196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:07.871566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:07.873026 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f7421-939b-45a0-b4b3-1f0d35a51925 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:07.884784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1036/2081] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:26:08.191550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a873364-2b44-4799-b860-753cd60e008c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.193149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2a873364-2b44-4799-b860-753cd60e008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:08.219068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a873364-2b44-4799-b860-753cd60e008c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.233583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.234525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.234976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.235442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] PATH_INFO: `/v3/projects/43bc7cf474fa4518bc9aa3102f707550` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.236840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bc7cf474fa4518bc9aa3102f707550)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.239557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.373872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bc7cf474fa4518bc9aa3102f707550', 'request_id': 'req-36c6c1a1-24e4-4534-9124-a380dfa08bd5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:08.380918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bc7cf474fa4518bc9aa3102f707550', 'request_id': 'req-36c6c1a1-24e4-4534-9124-a380dfa08bd5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:08.396875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bc7cf474fa4518bc9aa3102f707550', 'request_id': 'req-36c6c1a1-24e4-4534-9124-a380dfa08bd5'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:08.425818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] The token cache is being invalidate because project 43bc7cf474fa4518bc9aa3102f707550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:26:08.426402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:08.436222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-36c6c1a1-24e4-4534-9124-a380dfa08bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:08.438581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1046/2082] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:26:08 2022] DELETE /identity/v3/projects/43bc7cf474fa4518bc9aa3102f707550 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:26:08.445873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.446507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.447018 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.497964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:08.500509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:08.595269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1037/2083] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:08.601404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.602338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.602686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.628223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:08.629858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:08.647093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1047/2084] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:08.651543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.661649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fd36a-5ef5-420d-afce-b6c46c5deb90 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.673101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.673418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.673700 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.674353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.674638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:26:08.674940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.676674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.694897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.696218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1038/2085] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:26:08.706686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121de4-0f6e-4368-8a64-9dcf2bbfb856 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.712884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.713740 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.714172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.714689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.715622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.717989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.765152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1048/2086] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:26:08 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:26:08.769534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.783535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1980e-5b96-494b-a5bc-dfbb305fd57f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.850709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.851375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.851641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.851910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.852538 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.854480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.908581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.911389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1039/2087] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:26:08 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:26:08.928462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675da13-98cf-4c8c-92c3-365a3609bf02 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.935853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.937908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.938533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.939113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.941144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.943978 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.960964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1049/2088] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:26:08.965070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:08.975782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40ee27-6381-4c36-962d-80d5b3eb7f0d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:08.981380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:08.981832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:08.982184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:08.982534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:08.983492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:08.985179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080152-5d7e-436c-8992-f8aed47e183c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:08.997410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1040/2089] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:26:09.001480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:09.012015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bdd79-ecf4-4214-adc4-a8d837fd746b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:09.017525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:09.017525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:09.017729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:09.017927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] PATH_INFO: `/v3/projects/11dc460c9e2b400cb03c91c325835ecc/users/9270f87b87774882b212d16f4927f4c6/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:09.035039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11dc460c9e2b400cb03c91c325835ecc, user_id=9270f87b87774882b212d16f4927f4c6, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:09.037742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:09.058080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1050/2090] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:26:09 2022] PUT /identity/v3/projects/11dc460c9e2b400cb03c91c325835ecc/users/9270f87b87774882b212d16f4927f4c6/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:26:09.068998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e7ede-840c-4358-bcc9-b412b75d005d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:09.069594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e7ede-840c-4358-bcc9-b412b75d005d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:09.070083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e7ede-840c-4358-bcc9-b412b75d005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:09.118829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-823e7ede-840c-4358-bcc9-b412b75d005d None None] MFA Rules not processed for user `9270f87b87774882b212d16f4927f4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:09.120380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-823e7ede-840c-4358-bcc9-b412b75d005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:09.171736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:09.227230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1041/2091] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:26:09.236077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:09.238227 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:09.254774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:09.316856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25ba81-8815-45d2-87f5-86696486df38 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:09.331969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:09.333298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:09.333588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:09.333872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:09.346852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:09.350351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5abec-36fa-4734-be41-e460ea758593 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:09.367542 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1051/2092] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:26:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:26:19.220379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:19.359268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=20 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:37.022143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:37.022774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:37.023805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:37.044619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] MFA Rules not processed for user `213a6f956e5c48b4b45a6481e29db89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:37.045949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:37.130358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1042/2093] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:37.709495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d61de-10ec-4add-aaa8-0c57b8955f5f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:37.710080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d61de-10ec-4add-aaa8-0c57b8955f5f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:37.710080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d61de-10ec-4add-aaa8-0c57b8955f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:37.731441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-749d61de-10ec-4add-aaa8-0c57b8955f5f None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:37.732972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-749d61de-10ec-4add-aaa8-0c57b8955f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:37.797909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1052/2094] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:37.849668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:37.851828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:37.953214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9c1ac-502d-46d9-b2e3-c188aa06fcdb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:37.961734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:37.962371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:37.962645 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:37.962909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:37.975624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:37.977307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:37.988038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1043/2095] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:26:38.675787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9cc1-0624-4fdc-9ff4-c35b1cb186b9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:38.676125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9cc1-0624-4fdc-9ff4-c35b1cb186b9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:38.676907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a9cc1-0624-4fdc-9ff4-c35b1cb186b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:38.707511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d53a9cc1-0624-4fdc-9ff4-c35b1cb186b9 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:38.709372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d53a9cc1-0624-4fdc-9ff4-c35b1cb186b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:38.719944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1053/2096] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:26:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:38.738328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:38.753010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946dfb0-e559-4c35-8e76-a866f47b89bb service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:38.778553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:38.779562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:38.779844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:38.780113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:38.789311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:38.790953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:38.803108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1044/2097] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:26:39.134913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee7d2-3cb1-47ce-9c17-8aa69f106193 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:39.135301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee7d2-3cb1-47ce-9c17-8aa69f106193 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:39.135572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee7d2-3cb1-47ce-9c17-8aa69f106193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:39.162472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-807ee7d2-3cb1-47ce-9c17-8aa69f106193 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:39.163732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-807ee7d2-3cb1-47ce-9c17-8aa69f106193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:39.173045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1054/2098] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:39.185756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:39.200234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4ee7-0844-4054-b75b-e4759761e8d1 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:39.207292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:39.208234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:39.208816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:39.209401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:39.221942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:39.223854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:39.234127 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1045/2099] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:26:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 25 13:26:39.566730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:39.567045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:39.567365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:39.590469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:39.591715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:39.602364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1055/2100] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:26:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:39.616880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:39.628655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86ced3-6389-4a55-bec4-3b6afe212199 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:39.634612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:39.635507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:39.635789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:39.636057 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:39.645040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:39.646665 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f0b6-713e-4f00-8e20-40ea55a7556e service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:39.656112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1046/2101] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:26:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:26:41.958112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:41.958850 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:42.032137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725e288-6f69-4b8e-9af7-c44dbee6354a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:42.036699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:42.037337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:42.037616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:42.037890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] PATH_INFO: `/v3/users/27b2ed46bee948a68202e0d6b9b5bdcd` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:42.039066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b2ed46bee948a68202e0d6b9b5bdcd)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:42.041046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:42.097972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b2ed46bee948a68202e0d6b9b5bdcd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.111653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b2ed46bee948a68202e0d6b9b5bdcd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.118994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b2ed46bee948a68202e0d6b9b5bdcd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.119802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b2ed46bee948a68202e0d6b9b5bdcd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.120429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b2ed46bee948a68202e0d6b9b5bdcd'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.134355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1056/2102] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:26:41 2022] DELETE /identity/v3/users/27b2ed46bee948a68202e0d6b9b5bdcd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:26:42.138627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:42.139125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:42.139253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:42.160560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:42.161788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:42.214134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=100 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:42.228028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1047/2103] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:42.238066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:42.239523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:42.246569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:42.314736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73dae2-fada-48a5-9883-1cd792e0f96d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:42.318466 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:42.319078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:42.319380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:42.319644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:42.328925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:42.330348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f59fdf-62de-44c3-a20c-b7cfa4373e96 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:42.341289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1057/2104] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:26:42.549885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:42.551991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:42.566757 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1cc4f-56da-48d3-b636-8357756e80c6 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:42.583255 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:42.583925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:42.584823 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:42.585178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] PATH_INFO: `/v3/projects/74a1bdbf22c64613aa72ee2e046adae9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:42.586228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a1bdbf22c64613aa72ee2e046adae9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:42.588210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:42.683049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a1bdbf22c64613aa72ee2e046adae9', 'request_id': 'req-993b6a53-4d60-4920-ba61-23e784845f72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.690282 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a1bdbf22c64613aa72ee2e046adae9', 'request_id': 'req-993b6a53-4d60-4920-ba61-23e784845f72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.699735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a1bdbf22c64613aa72ee2e046adae9', 'request_id': 'req-993b6a53-4d60-4920-ba61-23e784845f72'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.714137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] The token cache is being invalidate because project 74a1bdbf22c64613aa72ee2e046adae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:26:42.714429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.715287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:26:42.716806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1048/2105] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:26:42 2022] DELETE /identity/v3/projects/74a1bdbf22c64613aa72ee2e046adae9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:26:48.812746 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:48.813226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:48.813319 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:48.833814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:26:48.835011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:48.894554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1058/2106] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:26:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:26:48.907088 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:26:48.908546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:26:48.985741 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b6a53-4d60-4920-ba61-23e784845f72 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:26:48.991217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:26:48.991821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:26:48.992096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:26:48.992367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:26:49.002288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:26:49.003935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459c8ed-12f1-4f6d-b5a9-13c81c47f2bb service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:26:49.012927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1049/2107] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:26:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:26:58.886435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=49 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:26:59.009710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:01.539937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:01.541480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:01.610547 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afc92b-6956-4a4c-b929-91527dd1962d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:01.615798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:01.616608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:01.616884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:01.617158 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] PATH_INFO: `/v3/users/9270f87b87774882b212d16f4927f4c6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:01.618373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9270f87b87774882b212d16f4927f4c6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:01.620343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:01.672498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9270f87b87774882b212d16f4927f4c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:01.686348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9270f87b87774882b212d16f4927f4c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:01.693942 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9270f87b87774882b212d16f4927f4c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:01.694930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9270f87b87774882b212d16f4927f4c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:01.695955 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9270f87b87774882b212d16f4927f4c6'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:01.712476 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1059/2108] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:27:01 2022] DELETE /identity/v3/users/9270f87b87774882b212d16f4927f4c6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:27:01.717153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:01.717622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:01.717736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:01.737288 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:01.738501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:01.799289 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1050/2109] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:01.812862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:01.815105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:01.878875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2b6c5-6c28-47ff-a61a-4fe89fba837d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:01.883359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:01.884001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:01.884316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:01.884671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:01.894543 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:01.896025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2732c-cfeb-49bd-b50c-f8f57e0a2e52 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:01.905727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1060/2110] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:27:02.246589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:02.248098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:02.261183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832d40-4414-4f76-a368-fb8ef2b2ccb4 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:02.265891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:02.266600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:02.266946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:02.267333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] PATH_INFO: `/v3/projects/11dc460c9e2b400cb03c91c325835ecc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:02.268529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11dc460c9e2b400cb03c91c325835ecc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:02.270473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:02.318730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11dc460c9e2b400cb03c91c325835ecc', 'request_id': 'req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:02.325185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11dc460c9e2b400cb03c91c325835ecc', 'request_id': 'req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:02.336686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11dc460c9e2b400cb03c91c325835ecc', 'request_id': 'req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:02.348586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] The token cache is being invalidate because project 11dc460c9e2b400cb03c91c325835ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:27:02.348873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:02.349539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e840c71d-3bf4-4059-a4c0-0f0a73f614d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:27:02.351071 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1051/2111] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:27:02 2022] DELETE /identity/v3/projects/11dc460c9e2b400cb03c91c325835ecc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:27:03.864941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:03.865513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:03.865513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:03.867058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1061/2112] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:27:03.873491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12825-0c51-405b-ad58-b436b080e328 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:03.873934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12825-0c51-405b-ad58-b436b080e328 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:03.874023 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12825-0c51-405b-ad58-b436b080e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:03.895115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-06c12825-0c51-405b-ad58-b436b080e328 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:03.896430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-06c12825-0c51-405b-ad58-b436b080e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:03.970763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1052/2113] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:03.983014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:03.984785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:04.011576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:04.044110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3007967-4fc5-47bb-a90c-bb02cc384bb9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:04.047982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:04.048627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.048924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.049190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.059583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:04.061008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ee5f5-7ec9-4a48-affa-a409d2747979 service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:04.071786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1062/2114] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:27:04.147954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.148240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.148547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.149937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1053/2115] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:27:04.154613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd969-26df-4c5c-b14c-2a1efd328276 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.154924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd969-26df-4c5c-b14c-2a1efd328276 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.155216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd969-26df-4c5c-b14c-2a1efd328276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.176091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5f3fd969-26df-4c5c-b14c-2a1efd328276 None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:04.177321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5f3fd969-26df-4c5c-b14c-2a1efd328276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:04.231004 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1063/2116] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:27:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:04.243053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:04.244924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:04.258859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abeb84-da66-442e-b938-0aa26b07730e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:04.262594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:04.263113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.263438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.263707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.270998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:04.272477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c9c8f-2e7d-4523-b4c0-d529d7209270 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:04.280309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1054/2117] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:27:04.371750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.372130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.372409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.373774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1064/2118] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:27:04.379674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.379994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.380301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.389325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:04.404652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:04.406345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:04.418517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1055/2119] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:27:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:04.431515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:04.433499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:04.450121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce3970-44bd-4afa-8146-7e9b98ce4a40 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:04.454985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:04.455619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:04.455893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:04.456172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:04.468311 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:04.469842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc744257-6a3d-405d-ae22-760f64acec0e service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:04.482196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1065/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:27:06.014361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:06.025046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1ecaf-a8b2-4330-86cc-d479cf90cf7a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:06.029528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:06.030144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:06.030426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:06.030733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:06.032763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:06.104664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:06.106380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:06.118985 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1056/2121] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:27:10.707339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d0c2-d662-4076-8188-e6da25155c12 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:10.707805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d0c2-d662-4076-8188-e6da25155c12 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:10.708614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d0c2-d662-4076-8188-e6da25155c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:10.730222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1ab3d0c2-d662-4076-8188-e6da25155c12 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:10.731438 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1ab3d0c2-d662-4076-8188-e6da25155c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:10.788103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1066/2122] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:10.802902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:10.804569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:10.877954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3e5cc-af70-4008-9ade-b3bbc88e779e service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:10.883914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:10.884534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:10.884810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:10.885077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:10.894433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:10.896111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:10.904563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1057/2123] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:27:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:27:20.343491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822276-797b-4ff6-a2d8-36131dbc810c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:20.343491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822276-797b-4ff6-a2d8-36131dbc810c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:20.343491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91822276-797b-4ff6-a2d8-36131dbc810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:20.364083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-91822276-797b-4ff6-a2d8-36131dbc810c None None] MFA Rules not processed for user `44a2023bb4dc49838b8d5db6a2f155d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:20.365787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91822276-797b-4ff6-a2d8-36131dbc810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:20.423822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1067/2124] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:27:20.900946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=64 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:25.701885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:25.703423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:25.800246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92478c5-487b-421e-9d3c-0146f0fefd02 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:25.806062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:25.806688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:25.806966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:25.807273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:25.809053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:25.821833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:25.823526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:25.837235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1058/2125] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:27:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:27:30.417482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:35.832073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=32 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:38.311113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8c45-c4c0-42f7-a1d9-69c5b697a481 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:38.311606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8c45-c4c0-42f7-a1d9-69c5b697a481 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:38.311711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8c45-c4c0-42f7-a1d9-69c5b697a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:38.331920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-db2c8c45-c4c0-42f7-a1d9-69c5b697a481 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:27:38.333119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-db2c8c45-c4c0-42f7-a1d9-69c5b697a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:27:38.340919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1068/2126] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:27:38.368932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:27:38.376573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3fcec-8b28-4c0f-b788-d513efe01768 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:27:38.381991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:27:38.382589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:27:38.382899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:27:38.383181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:27:38.391595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:27:38.393175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6b8e6-caa8-4c05-854f-4e4a1f8c3d99 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:27:38.401096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1059/2127] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:27:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:27:48.326585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:27:48.398161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:28:26.259690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2adc1-3178-4e9c-9525-d9efc03eff66 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:26.260265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2adc1-3178-4e9c-9525-d9efc03eff66 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:26.260265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2adc1-3178-4e9c-9525-d9efc03eff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:26.280594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b0e2adc1-3178-4e9c-9525-d9efc03eff66 None None] MFA Rules not processed for user `213a6f956e5c48b4b45a6481e29db89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:28:26.281894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b0e2adc1-3178-4e9c-9525-d9efc03eff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:28:26.348197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1069/2128] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:28:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:28:33.529291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:33.529801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:33.529801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:33.531431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1060/2129] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:28:33.559846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:33.560342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:33.560452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:33.582427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] MFA Rules not processed for user `a8112d8c2ea24bd286a5f139ba765f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:28:33.583923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:28:33.597317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1070/2130] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:28:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:28:33.597555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:28:33.599740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:28:33.630219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:28:33.631921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:28:33.645119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad53f1-cce0-4594-9ff8-b6b52c3037b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:28:33.651204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:28:33.651832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:33.652104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:33.652374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:33.661278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:28:33.662871 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:28:33.672546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1071/2131] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:28:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:28:33.681922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855ce5-af6b-4ef6-b324-723611b3bebc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:28:33.686492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:28:33.687123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:33.687433 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:33.687707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] PATH_INFO: `/v3/users/44a2023bb4dc49838b8d5db6a2f155d9` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:33.688945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a2023bb4dc49838b8d5db6a2f155d9)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:28:33.690874 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:28:33.738319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a2023bb4dc49838b8d5db6a2f155d9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:33.750731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a2023bb4dc49838b8d5db6a2f155d9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:33.751520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a2023bb4dc49838b8d5db6a2f155d9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:33.758589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a2023bb4dc49838b8d5db6a2f155d9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:33.759294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5536a419-cd5d-4561-a356-4443e7d8518e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a2023bb4dc49838b8d5db6a2f155d9'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:33.773777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1061/2132] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:28:33 2022] DELETE /identity/v3/users/44a2023bb4dc49838b8d5db6a2f155d9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:28:33.950297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:28:33.962433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3427f7-68be-4e9e-8a49-4cc9dacdf126 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:28:34.021670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:28:34.022324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:34.022602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:34.022868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] PATH_INFO: `/v3/projects/aaa45dccffd64489a30da96f93cf802f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:34.024218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa45dccffd64489a30da96f93cf802f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:28:34.026191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:28:34.075688 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa45dccffd64489a30da96f93cf802f', 'request_id': 'req-2b269ded-d42b-4521-a0d0-4688c613cbb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:34.081938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa45dccffd64489a30da96f93cf802f', 'request_id': 'req-2b269ded-d42b-4521-a0d0-4688c613cbb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:34.089510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa45dccffd64489a30da96f93cf802f', 'request_id': 'req-2b269ded-d42b-4521-a0d0-4688c613cbb3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:34.101888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] The token cache is being invalidate because project aaa45dccffd64489a30da96f93cf802f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:28:34.102406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:34.103251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:28:34.105214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1072/2133] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:28:33 2022] DELETE /identity/v3/projects/aaa45dccffd64489a30da96f93cf802f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:28:38.725822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:28:38.726317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:28:38.726410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:28:38.746619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] MFA Rules not processed for user `28e088c7bf2f41a397d41e8d3c2d7bae`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:28:38.747892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:28:38.804372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1062/2134] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:28:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:28:44.098406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:28:48.797411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:03.891045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:03.893419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:03.961338 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b269ded-d42b-4521-a0d0-4688c613cbb3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:03.965331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:03.965937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:03.966213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:03.966484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:03.968710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:04.041396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:04.042815 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:04.053645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1073/2135] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:06.731364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:06.733423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:06.801550 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac9b9a-798e-4906-aa93-1b45c6468f99 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:06.805785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:06.806395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:06.806701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:06.806960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:06.809073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:06.823711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:06.825176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:06.836748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1063/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:14.049053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=54 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:15.808577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:15.811388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:15.828984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbab96-26cf-4ddb-a98e-87908f80300e service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:15.836645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:15.837251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:15.837522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:15.837770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:15.839852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:15.906354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:15.908775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:15.919819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1074/2137] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:16.831460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=26 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:25.496553 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:25.505754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a500a-2f19-4fcc-b229-1756326c00d4 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:25.510356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:25.510960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:25.511267 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:25.511532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:25.513637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:25.587481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:25.588942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:25.599529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1064/2138] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:25.915141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=28 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:29.274827 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:29.277208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:29.290895 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8d073-02d1-447d-8af1-afe82dcd088e service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:29.295533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:29.296164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.296439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.296712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] PATH_INFO: `/v3/users/213a6f956e5c48b4b45a6481e29db89c` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.297857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213a6f956e5c48b4b45a6481e29db89c)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:29.299847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:29.347777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213a6f956e5c48b4b45a6481e29db89c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.359415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213a6f956e5c48b4b45a6481e29db89c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.366783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213a6f956e5c48b4b45a6481e29db89c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.367469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213a6f956e5c48b4b45a6481e29db89c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.368100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-227eae8b-a2d4-4aff-8c0e-299c4bccc534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213a6f956e5c48b4b45a6481e29db89c'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.383481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1075/2139] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:29:29 2022] DELETE /identity/v3/users/213a6f956e5c48b4b45a6481e29db89c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:29:29.560476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:29.572176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee0192-d7bc-41d3-bba4-0716cf3eac80 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:29.631692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:29.632363 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.632643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.632918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] PATH_INFO: `/v3/projects/db46e2a9752d428bbfff5da2ddb16e3a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.634073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db46e2a9752d428bbfff5da2ddb16e3a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:29.636056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:29.681779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db46e2a9752d428bbfff5da2ddb16e3a', 'request_id': 'req-bfd05d7b-1284-439f-89d6-35f113cd0cd8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.688094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db46e2a9752d428bbfff5da2ddb16e3a', 'request_id': 'req-bfd05d7b-1284-439f-89d6-35f113cd0cd8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.696603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db46e2a9752d428bbfff5da2ddb16e3a', 'request_id': 'req-bfd05d7b-1284-439f-89d6-35f113cd0cd8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.708819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] The token cache is being invalidate because project db46e2a9752d428bbfff5da2ddb16e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:29:29.709110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.710234 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-bfd05d7b-1284-439f-89d6-35f113cd0cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:29.711880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1065/2140] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:29:29 2022] DELETE /identity/v3/projects/db46e2a9752d428bbfff5da2ddb16e3a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:29.717210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.717500 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.717776 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.739099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:29.740353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:29.803428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1076/2141] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:29.811488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.811776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.812044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.831818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:29.833007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:29.841754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1066/2142] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:29.846726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:29.861346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeab133-cb16-46bd-867e-e05c95d4736c None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:29.869271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:29.870096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.870477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.870873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.871764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:29:29.872067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:29.873683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:29.885853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1077/2143] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:29:29.890465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:29.899722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e851e98-fcfa-428e-a04f-caa63716633b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:29.904384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:29.904978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:29.905262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:29.905524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:29.906147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:29.908070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:29.924044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:29.939551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1067/2144] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:29:29 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:29.943877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:29.954236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52f716-45aa-43ea-ad02-456274b3d6e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:30.013561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:30.014161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.014431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.014697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.015345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:30.017273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:30.025431 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:30.041921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1078/2145] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:29:29 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:30.046257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:30.056580 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab286666-0f62-4456-89d2-3d7aa4f97fd6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:30.061097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:30.061686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.061960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.062224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.062863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:30.064644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:30.074577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1068/2146] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:30.078625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:30.089465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c194e-ad76-4166-88de-375847f4fdb3 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:30.095310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:30.095914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.096194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.096493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.097174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:30.098882 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164834ec-d8f5-46ad-bb25-9d754e84f536 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:30.110032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1079/2147] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:30.114121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:30.124534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e7fe4-b460-4d55-835b-e0d69046468b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:30.130349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:30.130987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.131301 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.131594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] PATH_INFO: `/v3/projects/3f05293850ad4efc8e5a3eb06a838c7e/users/21c5dd6652a14bb29a08a2e7f46c7ac4/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.146931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f05293850ad4efc8e5a3eb06a838c7e, user_id=21c5dd6652a14bb29a08a2e7f46c7ac4, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:30.149626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:30.165147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1069/2148] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:29:30 2022] PUT /identity/v3/projects/3f05293850ad4efc8e5a3eb06a838c7e/users/21c5dd6652a14bb29a08a2e7f46c7ac4/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:30.176528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.176825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.177100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.206360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] MFA Rules not processed for user `21c5dd6652a14bb29a08a2e7f46c7ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:30.207610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:30.275491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1080/2149] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:30.287070 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:30.289148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:30.366657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60f7b-a71c-4a1f-b7b1-dd68cb9e0b55 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:30.372564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:30.373460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:30.373460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:30.373701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:30.382587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:30.384100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:30.394179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1070/2150] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:35.041581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:35.044120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:35.061560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead962e8-9597-4a67-b277-2bb75a098ae0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:35.068489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:35.069100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:35.069371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:35.069634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:35.071598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:35.132562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:35.134186 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fbdc5-729d-4bec-9ebb-6598386a7e3a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:35.143515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1081/2151] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:37.732490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:37.793861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b69e31-e990-47a6-b6c3-2035a295f099 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:37.798579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:37.799251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:37.799533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:37.799808 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] PATH_INFO: `/v3/users/21c5dd6652a14bb29a08a2e7f46c7ac4` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:37.800978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21c5dd6652a14bb29a08a2e7f46c7ac4)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:37.802954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:37.851133 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21c5dd6652a14bb29a08a2e7f46c7ac4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:37.852492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:37.866324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21c5dd6652a14bb29a08a2e7f46c7ac4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:37.867117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21c5dd6652a14bb29a08a2e7f46c7ac4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:37.874399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21c5dd6652a14bb29a08a2e7f46c7ac4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:37.875080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21c5dd6652a14bb29a08a2e7f46c7ac4'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:37.889201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1071/2152] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:29:37 2022] DELETE /identity/v3/users/21c5dd6652a14bb29a08a2e7f46c7ac4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:37.894015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:37.894499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:37.894616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:37.913823 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:37.915014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:37.974294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1082/2153] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:37.984250 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:37.985879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:38.053242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee36ab-ff19-4149-b8b6-9d9aedef0cd8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.057405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.058014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.058292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.058563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.068944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.070409 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efce23-ca7d-475c-a75d-50e142d67d69 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.080185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1072/2154] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:38.356197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.357837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:38.370243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65290100-8432-4b45-be6b-13378d0dbbcb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.375591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.376230 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.376506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.376775 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] PATH_INFO: `/v3/projects/3f05293850ad4efc8e5a3eb06a838c7e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.378072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f05293850ad4efc8e5a3eb06a838c7e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.379951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.381539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:38.433447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f05293850ad4efc8e5a3eb06a838c7e', 'request_id': 'req-5e270329-cb2f-41d2-8c4b-25a30b8664e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:38.440039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f05293850ad4efc8e5a3eb06a838c7e', 'request_id': 'req-5e270329-cb2f-41d2-8c4b-25a30b8664e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:38.450154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f05293850ad4efc8e5a3eb06a838c7e', 'request_id': 'req-5e270329-cb2f-41d2-8c4b-25a30b8664e4'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:38.460872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] The token cache is being invalidate because project 3f05293850ad4efc8e5a3eb06a838c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:29:38.461155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:38.461781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5e270329-cb2f-41d2-8c4b-25a30b8664e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:38.463859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1083/2155] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:29:38 2022] DELETE /identity/v3/projects/3f05293850ad4efc8e5a3eb06a838c7e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:38.469109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.469389 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.469660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.489878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:38.491073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:38.552957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1073/2156] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:38.560969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.561282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.561559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.584515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:38.586101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:38.595191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1084/2157] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:38.599533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.615198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e52e7-98fc-4525-b2a2-20646a561492 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.620144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.620880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.621251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.622813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.622813 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:29:38.624543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.625386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.638943 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1074/2158] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:29:38.643771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.653088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11281211-ef97-44a4-a6fd-8b5f83b8e88f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.658488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.659263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.659614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.659936 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.660633 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.662580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.697792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1085/2159] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:29:38 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:29:38.701902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.713512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dff17e-b438-4824-9c1e-86d937bb11e5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.764159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:38.771439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.772066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.772406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.772698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.773410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.775921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.799179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1075/2160] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:29:38 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:38.802906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.813465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a1759-2416-4f2d-b30e-d5bedf0dc37c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.818046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.818637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.818875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.819135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.819794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.821371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.831614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1086/2161] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:38.835755 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.845843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be98bb-2752-45a6-bfee-b8ac6351a354 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.850443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.851025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.851353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.851618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.852260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.853904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeff57-76b9-40a9-bf68-5bf3a46e5d2b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.865011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1076/2162] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:29:38.869423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:38.881072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e460-3dcb-49ba-98b4-49ac74f93c7b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:38.885813 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:38.886454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.886731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.887039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] PATH_INFO: `/v3/projects/178ce1143a774034937e2227adec7245/users/9ce49b8afc284bf89d54bb7a9fbe8284/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.902107 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=178ce1143a774034937e2227adec7245, user_id=9ce49b8afc284bf89d54bb7a9fbe8284, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:38.904798 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:38.920200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1087/2163] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:29:38 2022] PUT /identity/v3/projects/178ce1143a774034937e2227adec7245/users/9ce49b8afc284bf89d54bb7a9fbe8284/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:38.930765 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:38.931049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:38.931383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:38.959344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] MFA Rules not processed for user `9ce49b8afc284bf89d54bb7a9fbe8284`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:38.961024 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:39.028951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1077/2164] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:39.033506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.097734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7dd3-d3a9-4047-8b72-4393544e1c7a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.103420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.104023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.104322 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.104584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.105205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.107129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.142204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1088/2165] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:29:39 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:39.143575 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.155058 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab7df-7bd0-40b3-aa2f-1e75020e695e None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.213270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.213838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.214101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.214372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.215011 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.216986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.240587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1078/2166] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:29:39 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:39.244826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.255330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3b6c1-0de1-4f43-921c-6b9990bdbce6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.260249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.260851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.261129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.261396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.262042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.263701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.269629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:39.274738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1089/2167] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:39.279356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.289661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f9d98-8430-4776-b045-8e016275ee2a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.295547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.296180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.296458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.296730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] PATH_INFO: `/v3/projects/be35d1cf59514edaab963b72d86dba0f/users/63ba09a7641f4082882c83df34179700/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.312404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be35d1cf59514edaab963b72d86dba0f, user_id=63ba09a7641f4082882c83df34179700, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.315327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.330953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1079/2168] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:29:39 2022] PUT /identity/v3/projects/be35d1cf59514edaab963b72d86dba0f/users/63ba09a7641f4082882c83df34179700/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:39.335544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.345973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df551e-8c15-4f83-ab62-d5f3f0765615 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.410712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.411383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.411661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.411924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.412562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.414235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194ab03-1ea0-4dfb-9497-b8233957bd1c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.424454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1090/2169] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:39.428708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.440041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b095d5-2e7e-4c70-9143-53f0256ea616 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.444831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.445484 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.445759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.446027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] PATH_INFO: `/v3/projects/be35d1cf59514edaab963b72d86dba0f/users/63ba09a7641f4082882c83df34179700/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.448021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be35d1cf59514edaab963b72d86dba0f, user_id=63ba09a7641f4082882c83df34179700, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.450686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.461681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1080/2170] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:29:39 2022] PUT /identity/v3/projects/be35d1cf59514edaab963b72d86dba0f/users/63ba09a7641f4082882c83df34179700/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:29:39.472423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.472627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.472897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.501559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] MFA Rules not processed for user `63ba09a7641f4082882c83df34179700`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:39.502756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:39.580297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1091/2171] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:39.595942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:39.597942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:39.619287 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:39.664464 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bf8dd-e710-4b6d-9222-0f40dc50eef1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:39.668443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:39.669052 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:39.669327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:39.669597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:39.678829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:39.680308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235de07-9c92-4ca0-b576-fe2d5a2b5a75 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:39.689941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1081/2172] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:40.090807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:40.162360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e71e9-0e05-4f44-b4c2-276a5521ee3f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:40.168160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:40.168805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.169083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.169347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] PATH_INFO: `/v3/users/9ce49b8afc284bf89d54bb7a9fbe8284` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.170486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce49b8afc284bf89d54bb7a9fbe8284)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:40.172521 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:40.196493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:40.227474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce49b8afc284bf89d54bb7a9fbe8284'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.241896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce49b8afc284bf89d54bb7a9fbe8284'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.248988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce49b8afc284bf89d54bb7a9fbe8284'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.249753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce49b8afc284bf89d54bb7a9fbe8284'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.250456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce49b8afc284bf89d54bb7a9fbe8284'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.263638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1092/2173] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:29:40 2022] DELETE /identity/v3/users/9ce49b8afc284bf89d54bb7a9fbe8284 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:40.268211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.268692 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.268812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.287366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:40.288572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:40.349668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1082/2174] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:40.359454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:40.361095 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:40.421141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587c89-a54a-44be-be7c-d5b17c331ea1 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:40.425065 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:40.425664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.425940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.426202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.435553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:40.436974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:40.446812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1093/2175] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:40.560583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:40.563747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:40.577034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304934a-e40f-4250-bfcf-ae718f30aa96 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:40.580969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:40.581919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.581919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.582300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.584227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:40.595479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:40.597081 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:40.618656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1083/2176] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:40.735856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:40.737423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:40.763765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b43de-6ca3-468c-b9d2-e6ec5bb74828 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:40.769370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:40.770076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.770413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.770768 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] PATH_INFO: `/v3/projects/178ce1143a774034937e2227adec7245` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.772050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] RBAC: Authorizing `identity:delete_project(project_id=178ce1143a774034937e2227adec7245)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:40.774052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:40.822647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178ce1143a774034937e2227adec7245', 'request_id': 'req-47559cf0-d1fa-404a-8585-46f5c84bf99b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.829261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178ce1143a774034937e2227adec7245', 'request_id': 'req-47559cf0-d1fa-404a-8585-46f5c84bf99b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.842208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178ce1143a774034937e2227adec7245', 'request_id': 'req-47559cf0-d1fa-404a-8585-46f5c84bf99b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.853622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] The token cache is being invalidate because project 178ce1143a774034937e2227adec7245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:29:40.853908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.854537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.856093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1094/2177] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:29:40 2022] DELETE /identity/v3/projects/178ce1143a774034937e2227adec7245 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:40.860320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:40.871844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5c3-e783-46bd-81e0-17105aad4d3c service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:40.932353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061232e-1083-4926-a645-930967507e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:40.932988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061232e-1083-4926-a645-930967507e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:40.933262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061232e-1083-4926-a645-930967507e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:40.933528 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061232e-1083-4926-a645-930967507e4d admin admin] PATH_INFO: `/v3/users/63ba09a7641f4082882c83df34179700` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:40.934651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061232e-1083-4926-a645-930967507e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ba09a7641f4082882c83df34179700)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:40.936704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061232e-1083-4926-a645-930967507e4d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:40.995264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ba09a7641f4082882c83df34179700'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:40.997064 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:41.011917 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ba09a7641f4082882c83df34179700'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.012655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ba09a7641f4082882c83df34179700'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.020110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ba09a7641f4082882c83df34179700'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.020843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ba09a7641f4082882c83df34179700'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.049316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1084/2178] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:29:40 2022] DELETE /identity/v3/users/63ba09a7641f4082882c83df34179700 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:29:41.310054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:41.318946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:41.392198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:41.478932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47559cf0-d1fa-404a-8585-46f5c84bf99b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:41.572284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:41.572926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.573661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.573661 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.575641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:41.581942 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:41.583632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:41.610453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:41.614473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8dc0a-5bbf-4c76-8ad1-4ab847025da7 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:41.641764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1095/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:41.686579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061232e-1083-4926-a645-930967507e4d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:41.691969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:41.692619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.692964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.693239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] PATH_INFO: `/v3/projects/be35d1cf59514edaab963b72d86dba0f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.694507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be35d1cf59514edaab963b72d86dba0f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:41.696598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:41.744164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be35d1cf59514edaab963b72d86dba0f', 'request_id': 'req-4b5376e2-2752-4753-8a92-000e30ede608'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.751328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be35d1cf59514edaab963b72d86dba0f', 'request_id': 'req-4b5376e2-2752-4753-8a92-000e30ede608'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.760309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be35d1cf59514edaab963b72d86dba0f', 'request_id': 'req-4b5376e2-2752-4753-8a92-000e30ede608'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.772305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] The token cache is being invalidate because project be35d1cf59514edaab963b72d86dba0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:29:41.772670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.773357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4b5376e2-2752-4753-8a92-000e30ede608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:29:41.774889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1085/2180] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:29:41 2022] DELETE /identity/v3/projects/be35d1cf59514edaab963b72d86dba0f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:29:41.779854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.780154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.780429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.800008 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:41.801226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:41.864332 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1096/2181] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:41.872968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.873364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.873640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.895130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:41.896474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:41.904684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1086/2182] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:41.909066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:41.918679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69709cbc-b637-479b-9a56-b7dc8984d8e8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:41.923631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:41.924240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.924501 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.924772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.925411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:29:41.925707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:41.927345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:41.939714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1097/2183] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:29:41.944476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:41.954112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32790d31-32d6-4117-9565-0bf0aa028bde None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:41.960552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:41.961196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:41.961470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:41.961752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:41.962403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:41.964359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:41.993642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1087/2184] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:29:41 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:41.997851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:42.008263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7630a5-8f28-4b55-b6ca-4b00457e228c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:42.045288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:42.070853 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:42.071510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.071788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.072079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.072715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:42.074692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:42.098525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1098/2185] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:29:41 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:29:42.102747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:42.113790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3050-2cf8-46a7-95db-7326c8607ab7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:42.118565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:42.119207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.119489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.119760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.120413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:42.122119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:42.132661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1088/2186] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:29:42.136786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:42.147303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaf249-2626-4791-aa45-49442a85a364 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:42.152062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:42.152730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.153010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.153273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.153968 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:42.155628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdefee-d6a3-4196-a2df-b5c972a28efe admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:42.168029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1099/2187] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:29:42.172784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:42.183544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab07ca-6d24-4598-b915-2fafb7825fb0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:42.188526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:42.189291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.189634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.189975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] PATH_INFO: `/v3/projects/9f828e5dcd19461ea189eeffd0bf5146/users/87429c246c63482280c97d3dcc0cc985/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.204668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:42.209396 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f828e5dcd19461ea189eeffd0bf5146, user_id=87429c246c63482280c97d3dcc0cc985, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:42.212759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:42.229312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1089/2188] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:29:42 2022] PUT /identity/v3/projects/9f828e5dcd19461ea189eeffd0bf5146/users/87429c246c63482280c97d3dcc0cc985/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:29:42.241308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.241609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.241887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.272165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] MFA Rules not processed for user `87429c246c63482280c97d3dcc0cc985`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:42.273395 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:42.348538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1100/2189] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:42.361307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:42.363796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:42.466000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb12991-4f65-47e3-8a65-9aa7837a45fe admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:42.471439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:42.472182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:42.472614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:42.472982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:42.485331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:42.486961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4572d1a-29dd-4e98-b028-ee3aa6259ff1 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:42.497581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1090/2190] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:44.902326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:44.903940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:44.962040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653816d-5de0-4657-b73c-a35cccc6d3f6 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:44.966408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:44.967029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:44.967452 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:44.967727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:44.969549 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:44.980280 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:44.981826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b2785-06cf-45da-bfb5-2ba409b4cf7d service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:44.990071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1101/2191] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:45.131974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c0d08-50e7-40cf-accf-597d3af09ed3 None None] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.132534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c0d08-50e7-40cf-accf-597d3af09ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.132534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c0d08-50e7-40cf-accf-597d3af09ed3 None None] PATH_INFO: `/` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.134137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1091/2192] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:29:45.140534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc588f-ab97-4194-9362-1706d0b3d60b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.140821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc588f-ab97-4194-9362-1706d0b3d60b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.141089 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc588f-ab97-4194-9362-1706d0b3d60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.161358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f3dc588f-ab97-4194-9362-1706d0b3d60b None None] MFA Rules not processed for user `cd98a9aa326741fba8ff2360f8e87e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:45.162559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f3dc588f-ab97-4194-9362-1706d0b3d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:45.169413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1102/2193] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:45.181966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c0d08-50e7-40cf-accf-597d3af09ed3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:45.193749 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c0d08-50e7-40cf-accf-597d3af09ed3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:45.197587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:45.198198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.198479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.198751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.206465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:45.207910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b00dee-febb-46d4-b6eb-50e011909468 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:45.216221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1092/2194] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:45.304108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906271a9-5744-4b53-a2b4-3e5287c43594 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.304396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906271a9-5744-4b53-a2b4-3e5287c43594 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.304669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906271a9-5744-4b53-a2b4-3e5287c43594 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.306057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1103/2195] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:29:45.311635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.311924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.312193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.334228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:45.335855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:45.418878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1093/2196] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:29:45.430975 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906271a9-5744-4b53-a2b4-3e5287c43594 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:45.442385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906271a9-5744-4b53-a2b4-3e5287c43594 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:45.446064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:45.446792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.447075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.447380 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.458255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:45.459557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc171be7-222c-40f9-9367-8c3469b27155 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:45.471288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1104/2197] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:45.532082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:45.534111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:45.616025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a9c81-4b1d-407d-a163-cea26777d84c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:45.617457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:45.618178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:45.618460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:45.618730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:45.621180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:45.634473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:45.636335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:45.653951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1094/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:47.007349 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb904e6-402d-48c1-8d5f-3f61336faf90 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:47.007906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb904e6-402d-48c1-8d5f-3f61336faf90 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:47.007906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb904e6-402d-48c1-8d5f-3f61336faf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:47.029738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3eb904e6-402d-48c1-8d5f-3f61336faf90 None None] MFA Rules not processed for user `cea2bf367ebd492b8bcbf2ed6b17e747`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:29:47.031093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3eb904e6-402d-48c1-8d5f-3f61336faf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:47.055656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:47.089881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1105/2199] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:29:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:29:47.425053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:29:47.426509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:47.432911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:47.440859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044093e2-1d09-4c91-bf4c-c1070ec2e54a service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:29:47.444854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:29:47.445524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:29:47.445863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:29:47.446195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:29:47.448025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:29:47.457833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:29:47.459365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:29:47.468960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1095/2200] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:29:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:29:57.082678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:29:57.464806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:01.059585 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:01.060077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:01.060164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:01.083385 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:01.084700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:01.140391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1106/2201] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:30:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:01.190429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:01.191968 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:01.264510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83be46a-4caa-486c-96f5-b36464933ec1 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:01.270095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:01.270717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:01.270993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:01.271300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:01.279982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:01.281578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bdec1-c24c-4776-91ef-75904f60aa73 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:01.291939 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1096/2202] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:30:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:30:01.342670 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:01.421030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593bcb5-9d36-4209-a13b-1809e8c287b2 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:01.426359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:01.427019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:01.427321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:01.427626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] PATH_INFO: `/v3/users/87429c246c63482280c97d3dcc0cc985` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:01.428834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87429c246c63482280c97d3dcc0cc985)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:01.430849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:01.507481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87429c246c63482280c97d3dcc0cc985'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:01.522760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87429c246c63482280c97d3dcc0cc985'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:01.536178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87429c246c63482280c97d3dcc0cc985'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:01.559551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87429c246c63482280c97d3dcc0cc985'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:01.560375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87429c246c63482280c97d3dcc0cc985'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:01.583709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1107/2203] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:01 2022] DELETE /identity/v3/users/87429c246c63482280c97d3dcc0cc985 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:01.588643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:01.588990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:01.589335 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:01.621466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:01.623735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:01.723488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1097/2204] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:01.733254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:01.734860 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:01.770555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:01.811645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7163-3f67-48be-b84d-a85fb601b59d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:01.821222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:01.821982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:01.822324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:01.822663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:01.844804 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:01.846566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d33f92-45bc-48cd-8216-612451f28d49 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:01.871236 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1108/2205] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:02.285020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.286860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:02.305094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac57323-76c3-4630-b1cf-0b57cd1f5de8 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.317400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.318224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.318533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.318791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] PATH_INFO: `/v3/projects/9f828e5dcd19461ea189eeffd0bf5146` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.321262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f828e5dcd19461ea189eeffd0bf5146)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.323375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.407736 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f828e5dcd19461ea189eeffd0bf5146', 'request_id': 'req-e9dcd922-b5d2-4e94-811c-24ef957201c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:02.417969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f828e5dcd19461ea189eeffd0bf5146', 'request_id': 'req-e9dcd922-b5d2-4e94-811c-24ef957201c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:02.433837 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f828e5dcd19461ea189eeffd0bf5146', 'request_id': 'req-e9dcd922-b5d2-4e94-811c-24ef957201c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:02.445809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] The token cache is being invalidate because project 9f828e5dcd19461ea189eeffd0bf5146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:02.446170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:02.446891 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e9dcd922-b5d2-4e94-811c-24ef957201c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:02.453918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1098/2206] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:02 2022] DELETE /identity/v3/projects/9f828e5dcd19461ea189eeffd0bf5146 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:02.455659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.456110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.456305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.480706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:02.481842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:02.557934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1109/2207] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:02.565583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.565944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.566283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.616414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:02.619268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:02.634200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1099/2208] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:02.638316 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.654575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188875-c0b2-4119-831a-6d662670e9b0 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.659671 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.660506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.660785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.661057 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.661720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:30:02.662092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.663771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.682925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1110/2209] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:30:02.684029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.698446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2ba2f-6e3f-4179-9aab-2a4e5eaf9e3a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.709772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.710744 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.711021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.711323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.711971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.713912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.722618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:02.763477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1100/2210] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:02 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:02.766351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.780719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982d27a-759b-40b7-a56f-49e816027527 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.854755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.855419 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.855695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.859471 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.860233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.862470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.895198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1111/2211] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:02 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:02.898358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.910452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d8078-5fee-45e9-b147-552d0dba12f8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.926967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.927795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.928130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.928466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.929194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.930934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.954384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.954568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1101/2212] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:02.961181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f26f0-7d84-4faa-807f-aa2ac210aed6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:02.966260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:02.966954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:02.967343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:02.967807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:02.968534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:02.970378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecaec0-615e-446e-a7f8-5a8995057927 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:02.982711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1112/2213] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:02.986987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:02.998655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece364-a970-4009-97c8-6ef76330f23d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.003810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.004679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.007912 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.008243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] PATH_INFO: `/v3/projects/4e137e09ed2744a7b9a1d7f5054eb7ec/users/ad1d33de28cb4d30a3a8f981cdd2d8fb/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.028782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e137e09ed2744a7b9a1d7f5054eb7ec, user_id=ad1d33de28cb4d30a3a8f981cdd2d8fb, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.031923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.051260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1102/2214] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:02 2022] PUT /identity/v3/projects/4e137e09ed2744a7b9a1d7f5054eb7ec/users/ad1d33de28cb4d30a3a8f981cdd2d8fb/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:03.063137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.063450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.063735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.094163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] MFA Rules not processed for user `ad1d33de28cb4d30a3a8f981cdd2d8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:03.095714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:03.120920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:03.178707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1113/2215] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:30:03.183330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.255789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c2783-5031-4b58-83f0-1ec6550f45a4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.260508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.261226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.261562 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.261914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.262624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.264632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.298839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1103/2216] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:03.300726 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.311874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1736eb-ffd5-4962-bfaa-16db518c1c29 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.386584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.386584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.386863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.387156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.388137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.390212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.426920 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1114/2217] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:30:03.430731 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.446148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684a845-1993-469f-b2ba-9c2024a2dd4c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.451449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.452220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.452589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.452949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.453705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.455743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.470545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1104/2218] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:03.474856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.488159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b1a5a-a1e7-4e92-a1bd-8bbf1eefc7ba admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.494658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.495425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.495844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.496341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.497146 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.498901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b86d-5e6e-43ae-9352-6d8ce1605b0d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.513651 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1115/2219] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:03.517978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.540432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c9c-1572-40df-a675-d5d7e4c5f66b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.559829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.559829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.559829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.559829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] PATH_INFO: `/v3/projects/73a8e902b34c4c2fbc515c26dd5d77e2/users/5fa7f7543a83443e82ef05ee144e5b75/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.576639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a8e902b34c4c2fbc515c26dd5d77e2, user_id=5fa7f7543a83443e82ef05ee144e5b75, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.579847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.614551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1105/2220] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:03 2022] PUT /identity/v3/projects/73a8e902b34c4c2fbc515c26dd5d77e2/users/5fa7f7543a83443e82ef05ee144e5b75/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:03.627890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.627890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.627890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.677602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] MFA Rules not processed for user `5fa7f7543a83443e82ef05ee144e5b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:03.678941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:03.767215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1116/2221] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:03.768613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.855886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4016-7839-43de-b541-c3a546413600 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.864102 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.864733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.865068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.865338 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.865988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.867937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:03.872718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:03.902982 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1106/2222] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:03.907250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:03.922691 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f752fa9-4ed7-44ca-aca5-35e7ea72551b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:03.953763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:03.987446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:03.988957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:03.989334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:03.991179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:03.991832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:03.993851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.024944 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1117/2223] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:03 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:04.026522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:04.048621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c9404-f554-4874-a083-fa2fa4719964 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:04.058200 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:04.058843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.059117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.059439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.060153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:04.061780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.083532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1107/2224] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:04.085153 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:04.103116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bf8d8-4768-4074-8a07-9f7a0600a305 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:04.110368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:04.111513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.111889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.112253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] PATH_INFO: `/v3/projects/6837c2266e8147eca13f0f981c41b3cc/users/122d59cda4f3411ebff01637d9db1fa2/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.141812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6837c2266e8147eca13f0f981c41b3cc, user_id=122d59cda4f3411ebff01637d9db1fa2, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:04.145228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.176910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1118/2225] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:04 2022] PUT /identity/v3/projects/6837c2266e8147eca13f0f981c41b3cc/users/122d59cda4f3411ebff01637d9db1fa2/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:04.183087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:04.196537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77e31-582a-426f-bd75-5be2cda2d142 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:04.268498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:04.269317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.269847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.270235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.271036 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:04.273143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acab77-cc8d-49cd-8403-b588362b6b2f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.288428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1108/2226] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:04.289614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:04.301571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcc2af-71f2-43ef-8dd0-3efc5e6986e9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:04.306410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:04.307078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.307386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.307663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] PATH_INFO: `/v3/projects/6837c2266e8147eca13f0f981c41b3cc/users/122d59cda4f3411ebff01637d9db1fa2/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.309748 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6837c2266e8147eca13f0f981c41b3cc, user_id=122d59cda4f3411ebff01637d9db1fa2, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:04.313036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.324148 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1119/2227] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:04 2022] PUT /identity/v3/projects/6837c2266e8147eca13f0f981c41b3cc/users/122d59cda4f3411ebff01637d9db1fa2/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:04.334826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.335957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.336349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.375577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] MFA Rules not processed for user `122d59cda4f3411ebff01637d9db1fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:04.377146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:04.458089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1109/2228] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:04.470581 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:04.472350 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:04.540647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060054e9-171b-43ed-91ed-96802ec31128 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:04.544618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:04.545257 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:04.545536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:04.545809 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:04.628054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:04.630153 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:04.649781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1120/2229] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:05.756259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:05.760511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:05.843189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:05.873675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb291f2-440b-4328-99eb-6013354dec5c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:05.880483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:05.881391 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:05.881780 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:05.882141 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:05.884198 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:05.904987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:05.907745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:05.936488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1110/2230] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:06.588374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:06.589852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:06.601849 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbdcdb-cea9-47dc-a681-10d8a7843dd1 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:06.610572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:06.611265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:06.611550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:06.611821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:06.663055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:06.714566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:06.716161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:06.727010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1121/2231] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:07.473352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:07.484507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db678-4128-4697-bfaf-0c9eaa4cbe22 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:07.489016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:07.489797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:07.490278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:07.490831 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:07.509925 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:07.511779 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539657-1aaf-457c-aefd-65c9be3a4165 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:07.534352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1111/2232] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:11.754812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:11.897977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be03f-3fdc-4f5c-b3f9-54862fe2635c service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:11.908552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:11.908552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:11.908552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:11.910225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] PATH_INFO: `/v3/users/ad1d33de28cb4d30a3a8f981cdd2d8fb` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:11.911867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1d33de28cb4d30a3a8f981cdd2d8fb)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:11.914693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:11.980606 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1d33de28cb4d30a3a8f981cdd2d8fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:11.995432 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1d33de28cb4d30a3a8f981cdd2d8fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.002103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1d33de28cb4d30a3a8f981cdd2d8fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.003435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1d33de28cb4d30a3a8f981cdd2d8fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.004417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1d33de28cb4d30a3a8f981cdd2d8fb'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.041792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1122/2233] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:11 2022] DELETE /identity/v3/users/ad1d33de28cb4d30a3a8f981cdd2d8fb => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:12.043661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:12.044119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:12.044229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:12.069167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:12.070662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:12.154293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1112/2234] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:12.167681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:12.169141 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:12.255166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5222-d4ce-49de-97c8-51d10463fa84 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:12.271415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:12.271415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:12.271415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:12.271415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:12.283788 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:12.285406 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:12.289754 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:12.298595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1123/2235] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:30:12.605344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:12.608555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:12.630236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511b45d-99b4-4f63-98a8-c11858e0d7d2 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:12.636243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:12.636899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:12.637225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:12.637954 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] PATH_INFO: `/v3/projects/4e137e09ed2744a7b9a1d7f5054eb7ec` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:12.639139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e137e09ed2744a7b9a1d7f5054eb7ec)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:12.641576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:12.694375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e137e09ed2744a7b9a1d7f5054eb7ec', 'request_id': 'req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.701162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e137e09ed2744a7b9a1d7f5054eb7ec', 'request_id': 'req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.710950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e137e09ed2744a7b9a1d7f5054eb7ec', 'request_id': 'req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.723758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] The token cache is being invalidate because project 4e137e09ed2744a7b9a1d7f5054eb7ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:12.723990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.724691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.726486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1113/2236] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:12 2022] DELETE /identity/v3/projects/4e137e09ed2744a7b9a1d7f5054eb7ec => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:30:12.730401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:12.741178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfa596-0763-45d5-8f9e-9223692e3ecd service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:12.803999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:12.804733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:12.805013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:12.805288 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] PATH_INFO: `/v3/users/5fa7f7543a83443e82ef05ee144e5b75` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:12.806449 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa7f7543a83443e82ef05ee144e5b75)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:12.808462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:12.855762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa7f7543a83443e82ef05ee144e5b75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.870608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa7f7543a83443e82ef05ee144e5b75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.877716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa7f7543a83443e82ef05ee144e5b75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.878888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa7f7543a83443e82ef05ee144e5b75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.879624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa7f7543a83443e82ef05ee144e5b75'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:12.897127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1124/2237] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:12 2022] DELETE /identity/v3/users/5fa7f7543a83443e82ef05ee144e5b75 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:13.263596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:13.265336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:13.284105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02ded7-03d7-4aa7-8fdd-bc68b487f2cc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:13.341007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:13.355280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:13.356155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:13.356442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:13.356711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] PATH_INFO: `/v3/projects/73a8e902b34c4c2fbc515c26dd5d77e2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:13.357841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a8e902b34c4c2fbc515c26dd5d77e2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:13.359806 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:13.422155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a8e902b34c4c2fbc515c26dd5d77e2', 'request_id': 'req-ed6a6676-980e-467f-8bf7-6d6657e2ebce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.429634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a8e902b34c4c2fbc515c26dd5d77e2', 'request_id': 'req-ed6a6676-980e-467f-8bf7-6d6657e2ebce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.440512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a8e902b34c4c2fbc515c26dd5d77e2', 'request_id': 'req-ed6a6676-980e-467f-8bf7-6d6657e2ebce'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.452351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] The token cache is being invalidate because project 73a8e902b34c4c2fbc515c26dd5d77e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:13.452643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.453293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.454804 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1114/2238] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:13 2022] DELETE /identity/v3/projects/73a8e902b34c4c2fbc515c26dd5d77e2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:13.460870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:13.462534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:13.479733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2b35b-bd4b-40b0-8aa5-75eb26d31d4a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:13.538582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:13.539228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:13.539502 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:13.539767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] PATH_INFO: `/v3/users/122d59cda4f3411ebff01637d9db1fa2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:13.540902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122d59cda4f3411ebff01637d9db1fa2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:13.542994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:13.566474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:13.595769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122d59cda4f3411ebff01637d9db1fa2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.611744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122d59cda4f3411ebff01637d9db1fa2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.628683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122d59cda4f3411ebff01637d9db1fa2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.629891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122d59cda4f3411ebff01637d9db1fa2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.630283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3f600872-c08f-478f-883e-019df8b21d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122d59cda4f3411ebff01637d9db1fa2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:13.645665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1125/2239] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:13 2022] DELETE /identity/v3/users/122d59cda4f3411ebff01637d9db1fa2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:13.929734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:13.931451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:13.945885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6676-980e-467f-8bf7-6d6657e2ebce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.008675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.009310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.009584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.009850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] PATH_INFO: `/v3/projects/6837c2266e8147eca13f0f981c41b3cc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.010999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6837c2266e8147eca13f0f981c41b3cc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.012979 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.061953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6837c2266e8147eca13f0f981c41b3cc', 'request_id': 'req-85841578-86bf-457b-be09-30e5f1be7dc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:14.068655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6837c2266e8147eca13f0f981c41b3cc', 'request_id': 'req-85841578-86bf-457b-be09-30e5f1be7dc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:14.076798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6837c2266e8147eca13f0f981c41b3cc', 'request_id': 'req-85841578-86bf-457b-be09-30e5f1be7dc6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:14.087348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] The token cache is being invalidate because project 6837c2266e8147eca13f0f981c41b3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:14.087630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:14.088240 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-85841578-86bf-457b-be09-30e5f1be7dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:14.089735 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1115/2240] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:13 2022] DELETE /identity/v3/projects/6837c2266e8147eca13f0f981c41b3cc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:14.094829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.095327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.095420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.116077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:14.117261 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:14.178636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1126/2241] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:14.187854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137303-fcfe-4516-9199-18088b64664d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.188132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137303-fcfe-4516-9199-18088b64664d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.188398 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137303-fcfe-4516-9199-18088b64664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.209628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-57137303-fcfe-4516-9199-18088b64664d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:14.211214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-57137303-fcfe-4516-9199-18088b64664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:14.220582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1116/2242] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:14.225061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.235139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307837-f536-43a1-b1fd-2478a48723db None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.240378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.240976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.241254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.241528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.242179 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:30:14.242475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.244181 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.256379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1127/2243] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:30:14.260725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57137303-fcfe-4516-9199-18088b64664d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.266390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:14.272564 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57137303-fcfe-4516-9199-18088b64664d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.279378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.280095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.280369 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.280643 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.281308 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.283642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.315674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1117/2244] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:14 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:14.317163 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.327580 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9476e75-1604-42ff-a03f-3d5834feadc6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.389333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.389972 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.390253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.390520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.391213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.393162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.415270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1128/2245] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:14 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:14.424836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.435984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c5894-2223-4298-9624-c07d838bec38 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.440802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.441401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.441678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.441946 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.442589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.444257 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.454334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1118/2246] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:14.458496 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.469313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e262-0bd5-4a05-858c-908b1f5feb3c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.474213 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.474807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.475075 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.475399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.476174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.477795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa0e34-536b-491d-be97-98f695dbf694 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.489252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1129/2247] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:14.493534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.503647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0d1fe-7ed4-412c-b787-05bade69eefc admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.508536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.509262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.509607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.509963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] PATH_INFO: `/v3/projects/5ef76946984f47b399256903fcce6efe/users/b1fcb5599a424238a85f5c5ad8bd7640/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.525361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef76946984f47b399256903fcce6efe, user_id=b1fcb5599a424238a85f5c5ad8bd7640, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.528143 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.544451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1119/2248] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:14 2022] PUT /identity/v3/projects/5ef76946984f47b399256903fcce6efe/users/b1fcb5599a424238a85f5c5ad8bd7640/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:14.554961 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d574c-d100-437a-a4df-6135550ad053 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.555294 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d574c-d100-437a-a4df-6135550ad053 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.555566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d574c-d100-437a-a4df-6135550ad053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.563085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:14.590958 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5c3d574c-d100-437a-a4df-6135550ad053 None None] MFA Rules not processed for user `b1fcb5599a424238a85f5c5ad8bd7640`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:14.592325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5c3d574c-d100-437a-a4df-6135550ad053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:14.665995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1130/2249] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:14.677785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:14.679507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:14.737918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607dbca-b773-416a-b4b0-ec8bc1355321 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:14.743185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:14.743955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:14.744275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:14.744646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:14.753855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:14.755732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7165cf-a671-476d-ac48-7ee77f7e01e0 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:14.765832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1120/2250] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:16.808782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf538e-9fec-4adb-83a0-e8ca3f0363c7 None None] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:16.809330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf538e-9fec-4adb-83a0-e8ca3f0363c7 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:16.809330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf538e-9fec-4adb-83a0-e8ca3f0363c7 None None] PATH_INFO: `/` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:16.810653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1131/2251] 10.222.0.36 () {54 vars in 921 bytes} [Tue Oct 25 13:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 25 13:30:16.815656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76cc-4513-4e4f-b51e-3b715aba96e7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:16.816134 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76cc-4513-4e4f-b51e-3b715aba96e7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:16.816228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76cc-4513-4e4f-b51e-3b715aba96e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:16.837640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d5a76cc-4513-4e4f-b51e-3b715aba96e7 None None] MFA Rules not processed for user `b45a976b6672412f92f9a4f8fbc75b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:16.838890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d5a76cc-4513-4e4f-b51e-3b715aba96e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:16.919895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1121/2252] 10.222.0.36 () {58 vars in 1018 bytes} [Tue Oct 25 13:30:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:16.931785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf538e-9fec-4adb-83a0-e8ca3f0363c7 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:16.942656 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf538e-9fec-4adb-83a0-e8ca3f0363c7 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:16.946410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:16.946835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:16.947114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:16.947512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:16.958772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:16.960808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:16.972714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1132/2253] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:19.088489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3621-7575-4210-9bb4-7e571ee1e646 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:19.089059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3621-7575-4210-9bb4-7e571ee1e646 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:19.089059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3621-7575-4210-9bb4-7e571ee1e646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:19.109302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-77ec3621-7575-4210-9bb4-7e571ee1e646 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:19.110516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-77ec3621-7575-4210-9bb4-7e571ee1e646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:19.152767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:19.171077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1122/2254] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:30:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:19.183325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:19.184900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:19.265188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289ff5b-7107-4603-ac8a-6a49c9379e0d service glance] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:19.270188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:19.270795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:19.271069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:19.271365 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:19.280318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:19.282040 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:19.290991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1133/2255] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:30:29.164248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:29.287626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=69 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:40.535060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:40.535595 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:40.535762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:40.560447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:40.561640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:40.569191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1123/2256] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:40.581803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:40.593520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f983-5c28-4e47-aa6e-89e9165fafe0 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:40.599423 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:40.600023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:40.600301 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:40.600573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:40.609732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:40.611382 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32aa56-a74f-4124-b17a-cc7b1dbc5739 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:40.620113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1134/2257] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 25 13:30:43.870021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:43.938119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3b0e9-325c-4ab2-875c-4fe3e821dba3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:43.942931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:43.943588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:43.943864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:43.944154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] PATH_INFO: `/v3/users/b1fcb5599a424238a85f5c5ad8bd7640` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:43.945268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1fcb5599a424238a85f5c5ad8bd7640)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:43.947243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.000565 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1fcb5599a424238a85f5c5ad8bd7640'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.013871 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1fcb5599a424238a85f5c5ad8bd7640'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.014652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1fcb5599a424238a85f5c5ad8bd7640'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.022485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1fcb5599a424238a85f5c5ad8bd7640'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.023280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1fcb5599a424238a85f5c5ad8bd7640'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.039185 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1124/2258] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:43 2022] DELETE /identity/v3/users/b1fcb5599a424238a85f5c5ad8bd7640 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:44.044014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.044642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.044642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.064534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:44.065729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:44.132901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1135/2259] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:44.142620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:44.144140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:44.202723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d2f2b-6be9-4f0c-aa35-8baf6281f19b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:44.206868 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:44.207526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.207793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.208055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.218192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:44.219671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fcf80-8365-4be2-a568-39663dcd74c5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.229193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1125/2260] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:30:44.509122 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:44.510613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:44.523048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea46ff2-5123-4f57-b765-125f02e5f0cd None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:44.528177 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:44.528326 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.528623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.528890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] PATH_INFO: `/v3/projects/5ef76946984f47b399256903fcce6efe` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.530010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef76946984f47b399256903fcce6efe)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:44.531980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.580962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef76946984f47b399256903fcce6efe', 'request_id': 'req-bc95e5b6-96f2-4f33-a331-b1526cf58760'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.588513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef76946984f47b399256903fcce6efe', 'request_id': 'req-bc95e5b6-96f2-4f33-a331-b1526cf58760'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.597195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef76946984f47b399256903fcce6efe', 'request_id': 'req-bc95e5b6-96f2-4f33-a331-b1526cf58760'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.610403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] The token cache is being invalidate because project 5ef76946984f47b399256903fcce6efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:44.610689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.611430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-bc95e5b6-96f2-4f33-a331-b1526cf58760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:44.612985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1136/2261] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:44 2022] DELETE /identity/v3/projects/5ef76946984f47b399256903fcce6efe => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:44.618125 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.618410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.618678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.638582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:44.639738 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:44.649542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:44.699726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1126/2262] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:44.764844 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.765130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.765381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.787856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:44.788995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:44.798067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1137/2263] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:44.802410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:44.812277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf25ea8-2b4e-4fdb-aede-cbf12bb462e0 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:44.817325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:44.817934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.818209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.818477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.819269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:30:44.819567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:44.821157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.833182 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1127/2264] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:30:44.838081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:44.848058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e77d-e743-4725-84f6-3798bb5ef7c3 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:44.853475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:44.854082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.854356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.854623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.855291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:44.857199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.887757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1138/2265] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:44 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:44.892180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:44.902627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443007b7-ee9c-4bed-bcd2-769193d4d206 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:44.962445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:44.963076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:44.963390 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:44.963680 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:44.964343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:44.966378 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:44.993881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1128/2266] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:44 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:44.999481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:45.010193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a60d7-65d4-4e91-b07c-a88fc040d672 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:45.015424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:45.016036 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:45.016314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:45.016583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:45.017271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:45.018923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:45.029612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1139/2267] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:45.033655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:45.044089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac8350-82e0-4e26-b281-a996d9b37d0d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:45.049991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:45.050601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:45.050887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:45.051187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:45.051844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:45.053446 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e15b0-0b89-418e-bdbd-218a2b7473fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:45.064664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1129/2268] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:45.068797 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:45.079130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f344-23c8-4444-a6c1-09ce4fe3f4b5 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:45.084864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:45.085515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:45.086143 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:45.086421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] PATH_INFO: `/v3/projects/c96b57f990134149a2873a9eb5dcdb0e/users/f8ad48bfebf642d5899f011b26df1004/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:45.101604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96b57f990134149a2873a9eb5dcdb0e, user_id=f8ad48bfebf642d5899f011b26df1004, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:45.104298 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:45.119930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1140/2269] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:45 2022] PUT /identity/v3/projects/c96b57f990134149a2873a9eb5dcdb0e/users/f8ad48bfebf642d5899f011b26df1004/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:45.130283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:45.130573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:45.130846 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:45.159213 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] MFA Rules not processed for user `f8ad48bfebf642d5899f011b26df1004`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:45.160366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:45.229029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1130/2270] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:45.240102 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:45.241878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:45.252064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:45.305014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f87f79-261c-437d-9e72-ec29031f326b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:45.308771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:45.309370 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:45.309647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:45.309913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:45.318632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:45.320083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab75d-684d-4fc6-85f4-8712f3193f52 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:45.328886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1141/2271] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:55.226315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:55.325135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=23 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:55.727427 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:55.806845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d376ffb-210b-46d6-96fa-1c0440d4c362 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:55.812329 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:55.813012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:55.813327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:55.813637 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] PATH_INFO: `/v3/users/f8ad48bfebf642d5899f011b26df1004` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:55.814928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ad48bfebf642d5899f011b26df1004)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:55.817119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:55.867552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ad48bfebf642d5899f011b26df1004'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:55.879687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ad48bfebf642d5899f011b26df1004'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:55.880339 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ad48bfebf642d5899f011b26df1004'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:55.887535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ad48bfebf642d5899f011b26df1004'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:55.888226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ad48bfebf642d5899f011b26df1004'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:55.902031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1131/2272] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:30:55 2022] DELETE /identity/v3/users/f8ad48bfebf642d5899f011b26df1004 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:55.906795 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:55.907084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:55.907405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:55.926247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:55.927464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:55.986905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1142/2273] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:55.997273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:55.998797 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:56.059244 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d80b14-40fa-4ea4-bf4e-04b4a143d6a7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.063481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.064090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.064365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.064632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.074221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.075678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3a60-1313-4d93-bae2-1623954e780a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.085093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1132/2274] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:56.367473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.368992 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:56.381964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49e124-09af-4b53-89b7-8a9103df6e39 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.387012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.387713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.387954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.388241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] PATH_INFO: `/v3/projects/c96b57f990134149a2873a9eb5dcdb0e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.389383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96b57f990134149a2873a9eb5dcdb0e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.391469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.440077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96b57f990134149a2873a9eb5dcdb0e', 'request_id': 'req-a051d697-8a80-46fb-96fd-d9ebea8edc87'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:56.446274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96b57f990134149a2873a9eb5dcdb0e', 'request_id': 'req-a051d697-8a80-46fb-96fd-d9ebea8edc87'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:56.455721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96b57f990134149a2873a9eb5dcdb0e', 'request_id': 'req-a051d697-8a80-46fb-96fd-d9ebea8edc87'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:56.466997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] The token cache is being invalidate because project c96b57f990134149a2873a9eb5dcdb0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:30:56.467324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:56.467959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a051d697-8a80-46fb-96fd-d9ebea8edc87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:30:56.469469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1143/2275] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:30:56 2022] DELETE /identity/v3/projects/c96b57f990134149a2873a9eb5dcdb0e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:56.474673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.475168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.475278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.495893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:56.497083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:56.514746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:56.560167 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1133/2276] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:56.569462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.569863 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.570139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.591736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:56.592963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:56.602168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1144/2277] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:56.606632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.616790 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c55f-d645-446a-9f11-fd356fd4d702 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.622324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.622958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.623270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.623540 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.624214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:30:56.624512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.626129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.638318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1134/2278] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:30:56.643276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.652682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959b152-eb99-414c-a80b-da9a2536b598 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.657784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.658381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.658654 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.658922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.659579 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.661481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.689460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1145/2279] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:30:56 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:56.694000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.704326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff837a0-f1e3-4ad3-b230-895d55be283c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.761884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.762466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.762740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.763010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.763672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.765597 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.788977 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1135/2280] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:30:56 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:30:56.793624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.804202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca9b3-eab2-4ee6-886d-45e1aa74b67a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.808964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.809565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.809847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.810112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.810761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.812435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.822450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1146/2281] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:56.826487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.837370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9c401-b514-467b-a7ad-6d3bbbea15bf admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.842978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.843636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.843915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.844186 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.845525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.847243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2901a3c-bb55-4c4b-b499-9040be45244c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.858336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1136/2282] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:30:56.864512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:56.879344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0df34a-a235-4c41-ab67-2a884fb5e366 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:56.885050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:56.885700 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.885980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.886255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] PATH_INFO: `/v3/projects/e57dcb3164d740ecbfb764a1db6f94a3/users/3748cf65427f4d59a29574d6bb9f4c42/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.902535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57dcb3164d740ecbfb764a1db6f94a3, user_id=3748cf65427f4d59a29574d6bb9f4c42, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:56.905274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:56.925317 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1147/2283] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:30:56 2022] PUT /identity/v3/projects/e57dcb3164d740ecbfb764a1db6f94a3/users/3748cf65427f4d59a29574d6bb9f4c42/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:30:56.940941 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:56.941229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:56.941508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:56.970573 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] MFA Rules not processed for user `3748cf65427f4d59a29574d6bb9f4c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:30:56.971761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:57.038266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1137/2284] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:30:57.053389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:57.055515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:57.078985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:57.117277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ce429-7197-4ff3-beb9-e52f42e3032b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:57.121160 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:57.121762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:57.122034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:57.122299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:57.131489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:57.132935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:57.142194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1148/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:30:57.344739 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:30:57.346478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:57.367662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:30:57.442246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd532e-bafe-475d-92f0-8f07755eb720 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:30:57.448614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:30:57.449230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:30:57.449506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:30:57.449775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:30:57.451556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:30:57.512368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:30:57.514677 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e32d1-69b1-46b2-af88-577687328e4d service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:30:57.529027 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1138/2286] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:07.138344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=17 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:07.522447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=48 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:16.604277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:16.668534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14de707-2eef-4519-853c-75d3656993dd service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:16.673898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:16.674522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:16.674801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:16.675069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] PATH_INFO: `/v3/users/3748cf65427f4d59a29574d6bb9f4c42` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:16.676352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3748cf65427f4d59a29574d6bb9f4c42)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:16.678282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:16.727745 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3748cf65427f4d59a29574d6bb9f4c42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:16.742588 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3748cf65427f4d59a29574d6bb9f4c42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:16.749811 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3748cf65427f4d59a29574d6bb9f4c42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:16.751066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3748cf65427f4d59a29574d6bb9f4c42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:16.751708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3748cf65427f4d59a29574d6bb9f4c42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:16.766918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1149/2287] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:16 2022] DELETE /identity/v3/users/3748cf65427f4d59a29574d6bb9f4c42 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:31:16.771603 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:16.772107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:16.772107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:16.791256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:16.792548 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:16.859751 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1139/2288] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:16.869762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:16.871404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:16.930668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cab1e4-94e0-48c4-a195-fcb2df78c3ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:16.935192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:16.935816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:16.936091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:16.936356 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:16.946195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:16.947660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d0bc5-392e-4b3f-af83-f137927fae82 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:16.957144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1150/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:17.224602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.226106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:17.238657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541d4db-d518-4a57-afac-44da5172cc93 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.244623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.245273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.245551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.245820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] PATH_INFO: `/v3/projects/e57dcb3164d740ecbfb764a1db6f94a3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.247100 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57dcb3164d740ecbfb764a1db6f94a3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.249412 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.298311 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57dcb3164d740ecbfb764a1db6f94a3', 'request_id': 'req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:17.304510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57dcb3164d740ecbfb764a1db6f94a3', 'request_id': 'req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:17.313150 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57dcb3164d740ecbfb764a1db6f94a3', 'request_id': 'req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:17.329488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] The token cache is being invalidate because project e57dcb3164d740ecbfb764a1db6f94a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:17.329796 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:17.330547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-62d3451f-4b5e-4e5e-bb5b-e6d415e71977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:17.332269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1140/2290] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:17 2022] DELETE /identity/v3/projects/e57dcb3164d740ecbfb764a1db6f94a3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:17.337799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.338307 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.338545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.359120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:17.360353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:17.378402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:17.423971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1151/2291] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:17.437649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.438031 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.438394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.464097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:17.465343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:17.475330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1141/2292] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:17.480051 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.490526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8646-b522-41a2-9e0d-738afc541019 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.495924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.496527 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.496838 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.497088 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.497753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:31:17.498043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.499678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.511873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1152/2293] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:31:17.518381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.530750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b0500-950a-4c84-989a-0d4236482886 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.539511 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.540284 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.540632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.540974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.541934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.544424 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.577533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1142/2294] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:17 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:17.578771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.589276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec676e-0dff-422a-8078-445d9e15c55d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.647771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.648354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.648679 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.648949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.649591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.651550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.673859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1153/2295] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:17 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:17.678153 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.688758 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4227e0-4c0d-4cf8-874d-4995966ff7f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.693581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.694195 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.694455 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.694722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.695418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.697021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.709013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1143/2296] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:17.713673 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.724399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa704-56e7-4735-89ed-6ad99dcbc746 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.730814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.731563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.731837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.732105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.732750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.734352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629228c-7186-4fea-af11-f06192f9d4d2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.745247 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1154/2297] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:17.749512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.760299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbe4bb-8dbd-41a9-bdc6-9f60065dffce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.765723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:17.766490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.766824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.767192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] PATH_INFO: `/v3/projects/432b8c1f73ab4852b6e0dd03e4b49dce/users/638144d9f2c64c9a977d4231e4734f42/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.783888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b8c1f73ab4852b6e0dd03e4b49dce, user_id=638144d9f2c64c9a977d4231e4734f42, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:17.786535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:17.802130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1144/2298] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:17 2022] PUT /identity/v3/projects/432b8c1f73ab4852b6e0dd03e4b49dce/users/638144d9f2c64c9a977d4231e4734f42/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:17.812933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:17.813129 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:17.813451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:17.844090 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] MFA Rules not processed for user `638144d9f2c64c9a977d4231e4734f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:17.845976 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:17.921922 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1155/2299] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:17.933622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:17.935615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:17.959748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:17.995519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5cfd3-21fe-41ea-bfbd-e94d2d3d857b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:17.999606 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:18.000266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:18.000544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:18.000814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:18.009967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:18.011468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab5902-a76e-4dbd-aadb-233269f1ff50 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:18.020556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1145/2300] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:25.101140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:25.122908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:25.176473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b86f0-00c9-4016-9e85-47b9ab2afc2e None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:25.181802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:25.182484 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:25.182738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:25.183010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] PATH_INFO: `/v3/users/638144d9f2c64c9a977d4231e4734f42` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:25.184184 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=638144d9f2c64c9a977d4231e4734f42)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:25.186162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:25.235812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638144d9f2c64c9a977d4231e4734f42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.247948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638144d9f2c64c9a977d4231e4734f42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.255339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638144d9f2c64c9a977d4231e4734f42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.256274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638144d9f2c64c9a977d4231e4734f42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.257118 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638144d9f2c64c9a977d4231e4734f42'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.271123 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1156/2301] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:25 2022] DELETE /identity/v3/users/638144d9f2c64c9a977d4231e4734f42 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:25.275742 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:25.276243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:25.276341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:25.295505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:25.296713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:25.355776 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1146/2302] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:25.365582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:25.367172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:25.425960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005740a-f00a-4df1-b620-cafc62f3e5fb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:25.430002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:25.430611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:25.430888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:25.431188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:25.440493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:25.441927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:25.452018 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1157/2303] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:25.745628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:25.747136 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:25.760503 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0c14e-0aca-4713-b3a3-8eda679c4aa9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:25.765447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:25.766099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:25.766374 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:25.766642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] PATH_INFO: `/v3/projects/432b8c1f73ab4852b6e0dd03e4b49dce` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:25.767807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432b8c1f73ab4852b6e0dd03e4b49dce)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:25.769716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:25.817106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432b8c1f73ab4852b6e0dd03e4b49dce', 'request_id': 'req-509ec071-a51b-46d5-b4e8-2a5a3f00e023'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.823334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432b8c1f73ab4852b6e0dd03e4b49dce', 'request_id': 'req-509ec071-a51b-46d5-b4e8-2a5a3f00e023'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.832138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432b8c1f73ab4852b6e0dd03e4b49dce', 'request_id': 'req-509ec071-a51b-46d5-b4e8-2a5a3f00e023'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.843116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] The token cache is being invalidate because project 432b8c1f73ab4852b6e0dd03e4b49dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:25.843516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.844138 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:25.845670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1147/2304] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:25 2022] DELETE /identity/v3/projects/432b8c1f73ab4852b6e0dd03e4b49dce => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:31.873373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:31.874884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:31.886205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2dda3-75a2-4933-9583-d46a38f966f2 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:31.924021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:31.937634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:31.938251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:31.938524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:31.938790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:31.940703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:32.003686 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:32.005135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:32.014877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1158/2305] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:35.028253 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:35.030719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:35.088316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:35.112670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ec071-a51b-46d5-b4e8-2a5a3f00e023 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:35.118498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:35.119165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.119612 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.119933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.122077 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:35.135609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:35.137243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:35.156394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1148/2306] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:35.367693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:35.369315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:35.382525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560e08-405b-4964-a289-8143dc213b41 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:35.387930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:35.388600 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.388886 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.389146 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] PATH_INFO: `/v3/users/28e088c7bf2f41a397d41e8d3c2d7bae` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.390391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e088c7bf2f41a397d41e8d3c2d7bae)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:35.392545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:35.444404 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e088c7bf2f41a397d41e8d3c2d7bae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.457753 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e088c7bf2f41a397d41e8d3c2d7bae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.466016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e088c7bf2f41a397d41e8d3c2d7bae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.466784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e088c7bf2f41a397d41e8d3c2d7bae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.467413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-2c994f29-6307-4c2d-9177-44de91a790db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e088c7bf2f41a397d41e8d3c2d7bae'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.482251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1159/2307] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:35 2022] DELETE /identity/v3/users/28e088c7bf2f41a397d41e8d3c2d7bae => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:35.675660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:35.693155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c724f9-9337-4ff5-ba1b-73c34e786401 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:35.758838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:35.759545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.759846 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.760117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] PATH_INFO: `/v3/projects/230f3c911ac84f589b9b08b7d910e34d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.761218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230f3c911ac84f589b9b08b7d910e34d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:35.763205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:35.809226 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230f3c911ac84f589b9b08b7d910e34d', 'request_id': 'req-4856b1fc-b8ce-4615-86a9-096ae7b13298'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.816065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230f3c911ac84f589b9b08b7d910e34d', 'request_id': 'req-4856b1fc-b8ce-4615-86a9-096ae7b13298'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.825356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230f3c911ac84f589b9b08b7d910e34d', 'request_id': 'req-4856b1fc-b8ce-4615-86a9-096ae7b13298'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.835830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] The token cache is being invalidate because project 230f3c911ac84f589b9b08b7d910e34d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:35.836035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.837001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-4856b1fc-b8ce-4615-86a9-096ae7b13298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:35.838498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1149/2308] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:35 2022] DELETE /identity/v3/projects/230f3c911ac84f589b9b08b7d910e34d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:35.843678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.843963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.844235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.864576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:35.865820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:35.928064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1160/2309] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:35.936159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.936444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.936709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.957622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:35.958816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:35.968998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1150/2310] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:35.973848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:35.979058 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:35.990772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17644c-e762-4a00-ac68-8e256ab3a1e5 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:35.996234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:35.996858 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:35.997161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:35.997441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:35.998080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:31:35.998379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.000099 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.011983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1161/2311] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:31:36.016537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.025883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77fc6e-3283-4cfb-9e22-604bd05b1050 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.031712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.032347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.032566 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.032832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.033469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.035413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.065362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1151/2312] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:36.069613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.079913 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d7461-750c-45f7-aed3-f870bbcb6056 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.141368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.141896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.142175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.142451 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.143076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.145042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.168576 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1162/2313] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:31:36.172923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.183492 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659023c6-3d0b-4872-9ca7-7e377849564d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.188263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.188867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.189179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.189453 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.190108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.191830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.202549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1152/2314] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:36.206930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.218246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cbd7f-d2e1-4e98-aad6-7f5a8e616198 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.224987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.225571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.225846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.226113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.226763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.228442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b72bc4-8fce-4ebd-9e33-e9eecce15e49 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.240156 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1163/2315] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:36.244673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.255516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625c351-a8de-4333-9a45-7aa2a57a9fb4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.261414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.262054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.262336 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.262608 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] PATH_INFO: `/v3/projects/10aacab1a28c48a993db8e4fe1d799a6/users/d7958af7334a4d9ca60d8acbb2ea0c7d/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.278841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10aacab1a28c48a993db8e4fe1d799a6, user_id=d7958af7334a4d9ca60d8acbb2ea0c7d, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.281635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.299501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1153/2316] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:36 2022] PUT /identity/v3/projects/10aacab1a28c48a993db8e4fe1d799a6/users/d7958af7334a4d9ca60d8acbb2ea0c7d/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:36.311255 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.311545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.311817 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.340604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] MFA Rules not processed for user `d7958af7334a4d9ca60d8acbb2ea0c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:36.341814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:36.406048 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1164/2317] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:36.410542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.415218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:36.481658 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642de3a-5322-4b3f-816c-b66efd079255 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.489526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.489526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.489526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.489526 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.491137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.493129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.524622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1154/2318] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:36.526053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.538497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6f4e-3183-4649-9692-eeef76751bab None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.594896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:36.601762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.602393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.602885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.603366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.604206 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.606185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.634841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1165/2319] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/users => generated 308 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:36.638994 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.650096 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf125fed-2852-4d81-b58b-9a0894079a7b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.655449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.656217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.656497 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.656766 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.657451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.659117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.671168 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1155/2320] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:36.675602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.685966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238032-37f8-44e5-8257-1dddafc9f451 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.691738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.692384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.692663 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.692937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] PATH_INFO: `/v3/projects/2a44be9cded945138892f51f2763e30a/users/2e790ff81b484457ba6ebaa2b8f1d4a2/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.708598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a44be9cded945138892f51f2763e30a, user_id=2e790ff81b484457ba6ebaa2b8f1d4a2, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.711334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.726012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1166/2321] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:36 2022] PUT /identity/v3/projects/2a44be9cded945138892f51f2763e30a/users/2e790ff81b484457ba6ebaa2b8f1d4a2/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:36.730055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.740783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2202f6f-6152-4cdc-9392-4920cd141b41 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.799190 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.800083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.800321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.800581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.801216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.802820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c7552-cf65-4335-884f-ca154d6d1ee4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.812997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1156/2322] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:36.817557 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.828355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850aedb-f18d-4d92-8390-c12f50a8bb77 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:36.833161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:36.833816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.834096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.834368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] PATH_INFO: `/v3/projects/2a44be9cded945138892f51f2763e30a/users/2e790ff81b484457ba6ebaa2b8f1d4a2/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.836343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a44be9cded945138892f51f2763e30a, user_id=2e790ff81b484457ba6ebaa2b8f1d4a2, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:36.839403 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:36.849963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1167/2323] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:36 2022] PUT /identity/v3/projects/2a44be9cded945138892f51f2763e30a/users/2e790ff81b484457ba6ebaa2b8f1d4a2/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:36.860523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:36.860920 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:36.861228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:36.892928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] MFA Rules not processed for user `2e790ff81b484457ba6ebaa2b8f1d4a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:36.894604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:36.965730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:36.972294 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1157/2324] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:36.983034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:36.985074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:37.047590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798334c-1ebd-4328-a3d7-d764fa51ca8f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:37.051697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:37.052297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:37.052572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:37.052836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:37.064119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:37.065598 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528f6b4-6042-470c-8634-2fffc8d7b03c service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:37.078042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1168/2325] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:36 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:31:37.255880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:37.325965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2f479-fc15-443f-9408-ac6fd3577d6c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:37.331785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:37.332423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:37.332698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:37.332986 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] PATH_INFO: `/v3/users/d7958af7334a4d9ca60d8acbb2ea0c7d` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:37.334256 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7958af7334a4d9ca60d8acbb2ea0c7d)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:37.337212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:37.400967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7958af7334a4d9ca60d8acbb2ea0c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:37.415447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7958af7334a4d9ca60d8acbb2ea0c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:37.416254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7958af7334a4d9ca60d8acbb2ea0c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:37.424160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7958af7334a4d9ca60d8acbb2ea0c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:37.425105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7958af7334a4d9ca60d8acbb2ea0c7d'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:37.441357 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1158/2326] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:37 2022] DELETE /identity/v3/users/d7958af7334a4d9ca60d8acbb2ea0c7d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:37.446159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:37.446672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:37.446672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:37.467430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:37.468721 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:37.532081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1169/2327] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:37.544631 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:37.547210 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:37.615948 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4437a-3319-4511-94d5-5cb8a59bd8db admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:37.620121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:37.620908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:37.621191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:37.621444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:37.633911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:37.635476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:37.645921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1159/2328] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:37.708660 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:37.719921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bde15a-af6d-40f8-879b-357abee6c0ef None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:37.723832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:37.724538 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:37.724836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:37.725134 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:37.726931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:37.740274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:37.741715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:37.752183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1170/2329] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:31:37.988990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:37.990488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:38.006723 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d8ad-4253-4387-bb82-65f278902baa service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:38.013149 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:38.014630 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.014777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.015171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] PATH_INFO: `/v3/projects/10aacab1a28c48a993db8e4fe1d799a6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.016712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10aacab1a28c48a993db8e4fe1d799a6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:38.018708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:38.035500 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:38.073563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10aacab1a28c48a993db8e4fe1d799a6', 'request_id': 'req-5c45f73d-8931-44d0-b345-d9d39f798719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.080480 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10aacab1a28c48a993db8e4fe1d799a6', 'request_id': 'req-5c45f73d-8931-44d0-b345-d9d39f798719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.089971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10aacab1a28c48a993db8e4fe1d799a6', 'request_id': 'req-5c45f73d-8931-44d0-b345-d9d39f798719'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.101482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] The token cache is being invalidate because project 10aacab1a28c48a993db8e4fe1d799a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:38.101773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.102383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.103913 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1160/2330] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:37 2022] DELETE /identity/v3/projects/10aacab1a28c48a993db8e4fe1d799a6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:38.108132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:38.117864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c819dd-1c95-45fb-a6a9-527b4daa1856 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:38.155832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:38.175208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:38.175888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.176164 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.176433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] PATH_INFO: `/v3/users/2e790ff81b484457ba6ebaa2b8f1d4a2` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.177574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e790ff81b484457ba6ebaa2b8f1d4a2)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:38.179558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:38.231832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e790ff81b484457ba6ebaa2b8f1d4a2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.243831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e790ff81b484457ba6ebaa2b8f1d4a2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.252161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e790ff81b484457ba6ebaa2b8f1d4a2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.252908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e790ff81b484457ba6ebaa2b8f1d4a2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.253544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-6b3a2bf8-0a86-49d7-9412-5cacc11f8ba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e790ff81b484457ba6ebaa2b8f1d4a2'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.269701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1171/2331] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:38 2022] DELETE /identity/v3/users/2e790ff81b484457ba6ebaa2b8f1d4a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:31:38.543314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:38.545596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:38.560512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45f73d-8931-44d0-b345-d9d39f798719 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:38.627053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:38.627884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.628085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.628366 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] PATH_INFO: `/v3/projects/2a44be9cded945138892f51f2763e30a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.629634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a44be9cded945138892f51f2763e30a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:38.631517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:38.688579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a44be9cded945138892f51f2763e30a', 'request_id': 'req-ac126b7a-574d-4b09-b6c3-80e387ebb73a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.695220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a44be9cded945138892f51f2763e30a', 'request_id': 'req-ac126b7a-574d-4b09-b6c3-80e387ebb73a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.704297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a44be9cded945138892f51f2763e30a', 'request_id': 'req-ac126b7a-574d-4b09-b6c3-80e387ebb73a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.718158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] The token cache is being invalidate because project 2a44be9cded945138892f51f2763e30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:38.718582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.719230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-ac126b7a-574d-4b09-b6c3-80e387ebb73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:38.723241 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1161/2332] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:38 2022] DELETE /identity/v3/projects/2a44be9cded945138892f51f2763e30a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:38.726201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.726702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.726786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.749293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:38.750763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:38.817034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1172/2333] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:38.825111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.825451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.825602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.846364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:38.847695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:38.859508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1162/2334] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:38.864571 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:38.876083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0884b3-286d-47e4-808a-ebb722377f94 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:38.882980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:38.884000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.884486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.885021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.885739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:31:38.886038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:38.887674 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:38.903393 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1173/2335] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:31:38.905726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:38.915529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90108f91-45ce-4391-b4b7-4c364e42b394 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:38.920246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:38.921001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:38.921161 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:38.921404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:38.922312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:38.924316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:38.957724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1163/2336] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:38 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:31:38.962497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:38.974511 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956083-2ad6-4fce-a865-d648cdf5089f admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.024720 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:39.041006 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.041742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.042092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.042433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.043170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.045433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.072130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1174/2337] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:38 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:39.076290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.083956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:39.088887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedac601-4273-48a9-99f8-45bedcffacc0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.095910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.096955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.097545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.098291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.099085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.100910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.115079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1164/2338] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:39.116400 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.127100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7d53b-b301-43e4-b7b2-d4ffe2354d95 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.132737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.133354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.133628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.133892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.134544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.136283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0c839-f9e0-4646-bbca-f1234df86be7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.150718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1175/2339] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:39.154830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.168793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785e40-5e2c-409b-93d8-fe801c92bef5 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.174211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.174969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.175341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.175690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] PATH_INFO: `/v3/projects/75a436873877464d8208d5195c59ef8b/users/6e3f09cae3f848dcada3b50c485cbe66/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.194035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a436873877464d8208d5195c59ef8b, user_id=6e3f09cae3f848dcada3b50c485cbe66, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.196896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.213969 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1165/2340] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:39 2022] PUT /identity/v3/projects/75a436873877464d8208d5195c59ef8b/users/6e3f09cae3f848dcada3b50c485cbe66/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:39.229364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.229861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.230189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.264411 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] MFA Rules not processed for user `6e3f09cae3f848dcada3b50c485cbe66`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:39.267453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:39.339934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1176/2341] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:39.344489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.431915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376193fe-f3ac-4fb4-a0a7-846860a94bcb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.437350 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.438344 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.438812 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.439352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.440165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.442286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.487551 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1166/2342] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:39 2022] POST /identity/v3/projects => generated 357 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:39.491946 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.503555 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258ad9-4a9b-455e-a05c-ffd5337d699f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.588265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.589876 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.590343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.590676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.591562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.593867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.647515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1177/2343] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:39 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:39.647805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.659512 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6f18c-0701-42d0-adc8-4f468f8e5dd8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.667682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.668491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.668732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.669049 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.669784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.671715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.687918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1167/2344] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:31:39.691818 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.703835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baefd0b-ffca-4555-ac9b-5003084911af admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.709604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.710725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.711281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.712016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] PATH_INFO: `/v3/projects/6ad2ab85351242faaa27bab096f48c03/users/276d20f2c923444b938f4be67ec85679/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.731969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad2ab85351242faaa27bab096f48c03, user_id=276d20f2c923444b938f4be67ec85679, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.734997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.754333 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1178/2345] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:39 2022] PUT /identity/v3/projects/6ad2ab85351242faaa27bab096f48c03/users/276d20f2c923444b938f4be67ec85679/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:39.757863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.774660 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b510a-dc66-424d-95b4-d0ef41c3e1fa admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.855827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.856445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.856725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.856990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.857633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.859332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb43679-a9a3-4be1-a86d-a2b3effbf769 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.869678 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1168/2346] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:39.873548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:39.884441 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1985be-b5f8-47ac-b782-ae3ccecc6d8a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:39.889758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:39.890669 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.891446 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.891905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] PATH_INFO: `/v3/projects/6ad2ab85351242faaa27bab096f48c03/users/276d20f2c923444b938f4be67ec85679/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.894582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad2ab85351242faaa27bab096f48c03, user_id=276d20f2c923444b938f4be67ec85679, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:39.898388 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:39.910971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1179/2347] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:39 2022] PUT /identity/v3/projects/6ad2ab85351242faaa27bab096f48c03/users/276d20f2c923444b938f4be67ec85679/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:39.921203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:39.921501 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:39.921890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:39.951793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] MFA Rules not processed for user `276d20f2c923444b938f4be67ec85679`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:39.953207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:39.984786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:40.030517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1169/2348] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:40.042925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:40.045448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:40.061149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:40.119378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f5bb1-11fc-40ba-952d-acc8b51e5c99 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:40.122991 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:40.123639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:40.123918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:40.124185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:40.133422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:40.134868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:40.144640 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1180/2349] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:40.225238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:40.235063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e93b2-d9a0-48b3-b74a-80ad3143cf54 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:40.238687 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:40.239325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:40.239600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:40.239863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:40.306063 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:40.307582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:40.316845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1170/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:41.093638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.095618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:41.164041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0170ca-100d-461d-b014-f107c398a94f service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.168482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.169119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.169477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.169750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] PATH_INFO: `/v3/users/cea2bf367ebd492b8bcbf2ed6b17e747` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.170866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea2bf367ebd492b8bcbf2ed6b17e747)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.172841 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.222901 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea2bf367ebd492b8bcbf2ed6b17e747'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.234980 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea2bf367ebd492b8bcbf2ed6b17e747'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.241832 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea2bf367ebd492b8bcbf2ed6b17e747'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.242503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea2bf367ebd492b8bcbf2ed6b17e747'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.243115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-3008bb01-3a7c-412b-a8d7-bd46a537e75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea2bf367ebd492b8bcbf2ed6b17e747'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.257931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1181/2351] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:41 2022] DELETE /identity/v3/users/cea2bf367ebd492b8bcbf2ed6b17e747 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:31:41.435714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.447686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e86d87-7550-48b0-ab8b-40a9ba0c7083 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.508179 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.508832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.509109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.509377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] PATH_INFO: `/v3/projects/4029646841bb4bceb1dad1df19c4b870` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.510502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4029646841bb4bceb1dad1df19c4b870)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.512460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.565763 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4029646841bb4bceb1dad1df19c4b870', 'request_id': 'req-24640fe3-45a4-4bea-8f59-9a749d44a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.573147 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4029646841bb4bceb1dad1df19c4b870', 'request_id': 'req-24640fe3-45a4-4bea-8f59-9a749d44a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.580916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4029646841bb4bceb1dad1df19c4b870', 'request_id': 'req-24640fe3-45a4-4bea-8f59-9a749d44a2e7'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.586447 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:41.595577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] The token cache is being invalidate because project 4029646841bb4bceb1dad1df19c4b870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:41.595870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.597360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-24640fe3-45a4-4bea-8f59-9a749d44a2e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:41.599248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1171/2352] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:41 2022] DELETE /identity/v3/projects/4029646841bb4bceb1dad1df19c4b870 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:41.604492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.605166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.606046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.633191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:41.635170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:41.674417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:41.714843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1182/2353] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:41.724105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.724105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.724310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.744178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:41.745373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:41.753145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1172/2354] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:41.757650 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.768140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7eae-ab51-47db-aff3-bbf383f20bc8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.773353 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.774017 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.774296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.774565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.775258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:31:41.775564 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.777221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.788932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1183/2355] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:31:41.793361 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.802730 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccc06e-e69e-4864-9d6e-e1ca94411f55 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.807819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.808445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.808717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.808978 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.809605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.811554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.841568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1173/2356] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:41 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:31:41.845890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.856235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05085-8646-4354-bc70-00afb2edc091 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.856422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.857918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:41.916096 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.916708 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.916989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.917259 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.917891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.918092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dc243-c8d2-4b23-a221-db67efb3ed9b admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.919894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.922224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.922834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.923107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.923408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.925331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:41.942469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1184/2357] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:41 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:41.946644 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.956495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40f4dc-c6d3-4f68-9d6a-8ad429b7adcc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.961264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.962200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.962497 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.962767 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.963454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.965073 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.976251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1185/2358] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:31:41.979982 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:41.987681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:41.989721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:41.991780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22752881-27ee-4277-b6e7-1ff2605ce3ed admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:41.997653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:41.998310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:41.998652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:41.999118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1174/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:31:41.999266 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:41.999704 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:42.001433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540cc28-a1e5-4da0-a125-cd2f0f3d0ab0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:42.012344 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1186/2360] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:31:42.016615 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:42.027465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d963-077b-4574-9c8f-6baf506be763 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:42.032246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:42.032906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:42.033183 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:42.033448 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] PATH_INFO: `/v3/projects/7bedba73ec874058acaa1c78c3312e4b/users/75bf2318422245ee8bd06d541e0572fa/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:42.048938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bedba73ec874058acaa1c78c3312e4b, user_id=75bf2318422245ee8bd06d541e0572fa, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:42.051642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:42.066157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1175/2361] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:42 2022] PUT /identity/v3/projects/7bedba73ec874058acaa1c78c3312e4b/users/75bf2318422245ee8bd06d541e0572fa/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:42.071467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:42.071761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:42.072081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:42.091859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:42.093074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:42.156604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1187/2362] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:42.166662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:42.168674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:42.228000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb7b22-89cb-428a-9d9a-cfa2af2e1770 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:42.231897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:42.232493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:42.232767 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:42.233034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:42.242307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:42.243772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd35a17-e8de-4e35-a0cd-09d66773daf4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:42.247651 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:42.254486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1176/2363] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 25 13:31:43.095855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:43.098115 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:43.145943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:43.197902 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257ec4b-7190-4ce5-bab6-e828e1655444 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:43.204761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:43.205690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:43.206052 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:43.206420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:43.208590 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:43.223794 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:43.225635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:43.238450 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1188/2364] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:47.699561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:47.700083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:47.700320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:47.746305 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] MFA Rules not processed for user `75bf2318422245ee8bd06d541e0572fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:47.748048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:47.851211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1177/2365] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:47.865970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:47.877734 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885408b-fcd8-452b-91f1-0a3d16bed2a6 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:47.883659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:47.884267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:47.884552 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:47.884835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:47.895030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:47.896725 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba872fdc-f3a2-4bf5-9f87-fecc8b1fb25f service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:47.906323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1189/2366] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:48.615236 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:48.626539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab68f25-8a53-44c7-8842-32edaabdbdfc None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:48.631662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:48.632290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:48.632567 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:48.632833 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:48.634638 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:48.647180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:48.648415 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:48.659046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1178/2367] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:49.405672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6bbd-2778-4d51-b4c8-a3b454353fb9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:49.406315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6bbd-2778-4d51-b4c8-a3b454353fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:49.406315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6bbd-2778-4d51-b4c8-a3b454353fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:49.426098 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-8a1f6bbd-2778-4d51-b4c8-a3b454353fb9 None None] MFA Rules not processed for user `75bf2318422245ee8bd06d541e0572fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:49.427692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8a1f6bbd-2778-4d51-b4c8-a3b454353fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:49.436940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1190/2368] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:56.456709 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:56.458154 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:56.471904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9ab62-ce48-4737-9b87-201c08381f8d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:56.476859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:56.477535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:56.477800 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:56.478084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] PATH_INFO: `/v3/users/6e3f09cae3f848dcada3b50c485cbe66` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:56.479252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3f09cae3f848dcada3b50c485cbe66)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:56.481225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:56.532786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3f09cae3f848dcada3b50c485cbe66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:56.547355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3f09cae3f848dcada3b50c485cbe66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:56.548094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3f09cae3f848dcada3b50c485cbe66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:56.556821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3f09cae3f848dcada3b50c485cbe66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:56.557533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3f09cae3f848dcada3b50c485cbe66'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:56.575734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1179/2369] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:56 2022] DELETE /identity/v3/users/6e3f09cae3f848dcada3b50c485cbe66 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:56.580112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:56.580603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:56.580706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:56.600820 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:56.602345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:56.704914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1191/2370] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:56.715426 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:56.716951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:56.787237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:56.792773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2669c03-23ef-4fdc-8986-90b77b489ae9 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:56.797664 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:56.798544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:56.799221 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:56.799715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:56.809661 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:56.811121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:56.821684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1180/2371] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:31:57.429839 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:57.431461 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:57.454144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a69e3f-1108-450d-ab0c-b5e3e4684bc8 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:57.465890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:57.468716 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:57.469001 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:57.469274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] PATH_INFO: `/v3/projects/75a436873877464d8208d5195c59ef8b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:57.470977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a436873877464d8208d5195c59ef8b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:57.473347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:57.538055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:57.539536 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:57.564873 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea887d-92bf-4f81-a163-4d37ecddd4d4 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:57.580883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a436873877464d8208d5195c59ef8b', 'request_id': 'req-0475e1d6-ea97-4983-b008-b9ff5a111216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.590743 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a436873877464d8208d5195c59ef8b', 'request_id': 'req-0475e1d6-ea97-4983-b008-b9ff5a111216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.603020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a436873877464d8208d5195c59ef8b', 'request_id': 'req-0475e1d6-ea97-4983-b008-b9ff5a111216'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.620330 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] The token cache is being invalidate because project 75a436873877464d8208d5195c59ef8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:57.620906 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.621715 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.623695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1192/2372] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:57 2022] DELETE /identity/v3/projects/75a436873877464d8208d5195c59ef8b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:57.628113 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:57.630366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:57.643748 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:57.644477 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:57.644815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:57.647202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] PATH_INFO: `/v3/users/03a29b46be0145418eacb6d615f8499a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:57.647202 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a29b46be0145418eacb6d615f8499a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:57.648422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:57.651038 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:57.704208 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a29b46be0145418eacb6d615f8499a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.717788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a29b46be0145418eacb6d615f8499a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.718770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a29b46be0145418eacb6d615f8499a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.721093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475e1d6-ea97-4983-b008-b9ff5a111216 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:57.728093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a29b46be0145418eacb6d615f8499a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.729115 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a29b46be0145418eacb6d615f8499a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.729556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:57.730212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:57.730506 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:57.730759 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] PATH_INFO: `/v3/users/276d20f2c923444b938f4be67ec85679` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:57.732415 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=276d20f2c923444b938f4be67ec85679)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:57.735083 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:57.746864 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1181/2373] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:57 2022] DELETE /identity/v3/users/03a29b46be0145418eacb6d615f8499a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:57.802467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276d20f2c923444b938f4be67ec85679'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.819314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276d20f2c923444b938f4be67ec85679'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.827042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276d20f2c923444b938f4be67ec85679'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.827951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276d20f2c923444b938f4be67ec85679'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.828642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276d20f2c923444b938f4be67ec85679'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:57.847782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1193/2374] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:57 2022] DELETE /identity/v3/users/276d20f2c923444b938f4be67ec85679 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:58.043165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.044693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.122843 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8558-52ed-4733-b89e-e094fd62ea21 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.136160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:58.183634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.184269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.184545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.184810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] PATH_INFO: `/v3/projects/4bc21dd502ee4d349f1e0f0ce3a9ed3b` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.185971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc21dd502ee4d349f1e0f0ce3a9ed3b)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.187957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.203892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.205377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.224133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6f395-a111-4811-8376-ee123ffcb37d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.230249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.232050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.232512 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.233561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] PATH_INFO: `/v3/projects/6ad2ab85351242faaa27bab096f48c03` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.234851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ad2ab85351242faaa27bab096f48c03)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.237323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.245435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc21dd502ee4d349f1e0f0ce3a9ed3b', 'request_id': 'req-7c2ad8f7-f36e-4006-9fee-69138be8df13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.252867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc21dd502ee4d349f1e0f0ce3a9ed3b', 'request_id': 'req-7c2ad8f7-f36e-4006-9fee-69138be8df13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.261931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc21dd502ee4d349f1e0f0ce3a9ed3b', 'request_id': 'req-7c2ad8f7-f36e-4006-9fee-69138be8df13'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.273079 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] The token cache is being invalidate because project 4bc21dd502ee4d349f1e0f0ce3a9ed3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:58.273372 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.273984 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.275525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1182/2375] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:58 2022] DELETE /identity/v3/projects/4bc21dd502ee4d349f1e0f0ce3a9ed3b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:58.279508 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.281030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.287737 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ad2ab85351242faaa27bab096f48c03', 'request_id': 'req-24fadc6a-be7c-4a5a-aaed-27c7908e8344'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.294884 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ad2ab85351242faaa27bab096f48c03', 'request_id': 'req-24fadc6a-be7c-4a5a-aaed-27c7908e8344'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.303469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ad2ab85351242faaa27bab096f48c03', 'request_id': 'req-24fadc6a-be7c-4a5a-aaed-27c7908e8344'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.314561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] The token cache is being invalidate because project 6ad2ab85351242faaa27bab096f48c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:58.314834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.315499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-24fadc6a-be7c-4a5a-aaed-27c7908e8344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.317203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1194/2376] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:58 2022] DELETE /identity/v3/projects/6ad2ab85351242faaa27bab096f48c03 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:31:58.323121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fcf1-5fd8-496e-aaf8-fb7836b20e68 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.323486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fcf1-5fd8-496e-aaf8-fb7836b20e68 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.323763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fcf1-5fd8-496e-aaf8-fb7836b20e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.344528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-2376fcf1-5fd8-496e-aaf8-fb7836b20e68 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:58.345727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-2376fcf1-5fd8-496e-aaf8-fb7836b20e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.350173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ad8f7-f36e-4006-9fee-69138be8df13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.355219 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:58.397087 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:58.407417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1195/2377] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:58.416152 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.416470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.416762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.416877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.417050 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.417163 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.417337 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] PATH_INFO: `/v3/users/d359a6c2303d4bf7a91fe7ace3a43f48` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.418488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d359a6c2303d4bf7a91fe7ace3a43f48)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.420463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.437003 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:58.438270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.447026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1196/2378] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:31:58.451378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.461243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ae786-41bb-45e3-83c3-9f00b2b1187a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.466877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.467589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.467930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.468299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.468983 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:31:58.469291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.470891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.474054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd359a6c2303d4bf7a91fe7ace3a43f48'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.486428 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1197/2379] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:31:58.487325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.494907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd359a6c2303d4bf7a91fe7ace3a43f48'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.496181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd359a6c2303d4bf7a91fe7ace3a43f48'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.497647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daec63-28fa-4f68-ba25-eb4e2378f4bb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.504602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.504810 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd359a6c2303d4bf7a91fe7ace3a43f48'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.505231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.505532 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.505788 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd359a6c2303d4bf7a91fe7ace3a43f48'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.506046 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.507191 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.508664 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.521773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1183/2380] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:31:58 2022] DELETE /identity/v3/users/d359a6c2303d4bf7a91fe7ace3a43f48 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:31:58.543718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1198/2381] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:31:58 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:31:58.544851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.546113 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.620463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4c598-6122-43fc-ba54-8af5f2a2e4e8 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.693223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.693899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.694178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.694475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.695332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.697348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.711183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.713435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.726524 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b14779-2a0f-431a-8704-316ba8b4cf6b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.731708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1184/2382] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:31:58 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:31:58.732645 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.733290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.733570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.733874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] PATH_INFO: `/v3/projects/3063b248020e46bbaaca247b1387027d` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.735488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.735885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3063b248020e46bbaaca247b1387027d)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.737851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.746908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882e80f-58e4-452b-b621-43605d3a71ce admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.755061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.755710 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.755988 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.756262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.756926 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.758541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.760820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:58.770212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1185/2383] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:31:58.774499 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.785105 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f66b1-e6f6-4b53-9fc1-ff53b927a66c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.793429 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3063b248020e46bbaaca247b1387027d', 'request_id': 'req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.800341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3063b248020e46bbaaca247b1387027d', 'request_id': 'req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.807899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3063b248020e46bbaaca247b1387027d', 'request_id': 'req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.818627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] The token cache is being invalidate because project 3063b248020e46bbaaca247b1387027d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:31:58.819642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.819642 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:31:58.821135 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1199/2384] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:31:58 2022] DELETE /identity/v3/projects/3063b248020e46bbaaca247b1387027d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:58.848974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.849928 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.850209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.850479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.851206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.852904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64af1cf-6c3e-4978-b93a-d7e2e1e60850 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.864721 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1186/2385] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:31:58.869320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:58.870988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:58.885377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9e2b5-248a-4a36-aa6f-03ffe55aec1e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:58.891281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:58.891997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.892269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.892536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] PATH_INFO: `/v3/projects/2430cc9154554e2eb86df78be457a556/users/f2fc0e366b8946f0afc3a10459a93444/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.910932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2430cc9154554e2eb86df78be457a556, user_id=f2fc0e366b8946f0afc3a10459a93444, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:58.913662 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:58.929309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1200/2386] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:31:58 2022] PUT /identity/v3/projects/2430cc9154554e2eb86df78be457a556/users/f2fc0e366b8946f0afc3a10459a93444/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:31:58.940784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:58.941145 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:58.941518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:58.973726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] MFA Rules not processed for user `f2fc0e366b8946f0afc3a10459a93444`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:31:58.975060 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:59.100164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1187/2387] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:31:59.111100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:31:59.116632 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:31:59.154994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:31:59.180224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86559ae6-1af3-44a3-9b31-0c2fa779f372 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:31:59.183855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:31:59.184462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:31:59.184738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:31:59.185007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:31:59.193225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:31:59.194668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6495bd-ffbb-40ec-ab22-9a7028aafa5a service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:31:59.204457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1201/2388] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:32:02.266518 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:02.330906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df0d08-d443-4709-92e9-73b76516944d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:02.335949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:02.336577 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:02.336851 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:02.337482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] PATH_INFO: `/v3/users/f2fc0e366b8946f0afc3a10459a93444` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:02.338620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fc0e366b8946f0afc3a10459a93444)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:02.340601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:02.383659 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:02.403759 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fc0e366b8946f0afc3a10459a93444'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:02.415489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fc0e366b8946f0afc3a10459a93444'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:02.416157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fc0e366b8946f0afc3a10459a93444'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:02.423644 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fc0e366b8946f0afc3a10459a93444'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:02.424359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fc0e366b8946f0afc3a10459a93444'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:02.441531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1188/2389] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:32:02 2022] DELETE /identity/v3/users/f2fc0e366b8946f0afc3a10459a93444 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:32:02.447584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:02.448067 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:02.448171 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:02.467974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:02.469174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:02.553773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1202/2390] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:32:02.565594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:02.567315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:02.642642 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1886dfe-3500-45c1-896f-12352965780e admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:02.646881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:02.647614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:02.647965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:02.648306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:02.658177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:02.659668 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1b29d-e000-485f-afce-9773a0f4ec06 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:02.670140 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1189/2391] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:32:02.944773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:02.946210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:02.961012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e5b93-49d2-435d-8221-732b45dfe6bc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:02.967846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:02.968442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:02.968729 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:02.968977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] PATH_INFO: `/v3/projects/2430cc9154554e2eb86df78be457a556` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:02.970357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2430cc9154554e2eb86df78be457a556)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:02.972233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.021893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2430cc9154554e2eb86df78be457a556', 'request_id': 'req-d1f46554-ec97-4fc4-99b1-e5c44db4170a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:03.028169 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2430cc9154554e2eb86df78be457a556', 'request_id': 'req-d1f46554-ec97-4fc4-99b1-e5c44db4170a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:03.036325 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2430cc9154554e2eb86df78be457a556', 'request_id': 'req-d1f46554-ec97-4fc4-99b1-e5c44db4170a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:03.046799 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] The token cache is being invalidate because project 2430cc9154554e2eb86df78be457a556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:32:03.047081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:03.047732 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d1f46554-ec97-4fc4-99b1-e5c44db4170a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:03.049269 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1203/2392] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:32:02 2022] DELETE /identity/v3/projects/2430cc9154554e2eb86df78be457a556 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:32:03.054323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.054609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.054879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.074849 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:03.076117 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:03.140430 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1190/2393] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:32:03.148714 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.148996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.149268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.169566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:03.170802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:03.179376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1204/2394] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:03.183662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.193947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e588e1-8428-4f63-90e6-a714134202ca None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.199299 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.199805 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.200090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.200364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.201016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:32:03.201316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.202940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.215173 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1191/2395] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:32:03.219605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.229282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c6fc8-21fc-4383-bedf-132734227afc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.234563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.235196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.235505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.235846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.236486 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.238456 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.267831 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1205/2396] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:32:03 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:32:03.272242 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.282636 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402e7b5-74c0-45b5-8364-c07c879e0326 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.314066 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:03.343531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.344184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.344436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.344708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.345576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.347561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.369457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1192/2397] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:32:03 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:32:03.373639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.384465 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2301-7c7a-41b8-b53b-a960038327ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.390926 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.391573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.391845 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.392202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.392840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.394460 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.405548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1206/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:32:03.410343 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.420260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf12542-02ea-4784-8841-e1af24cbfc95 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.425359 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.425929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.426204 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.426471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.427110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.428740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d51689-333c-4e2c-b865-bc256c5115a8 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.438463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1193/2399] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:32:03.442993 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.458870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5e023-1fc9-48cd-b986-b50a29588070 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.467376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.468355 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.468757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.469133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] PATH_INFO: `/v3/projects/1ac4897af78b42bf8dd8edd9be89ff60/users/42f9afce22be463b98c3e0bd8296dc18/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.485203 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac4897af78b42bf8dd8edd9be89ff60, user_id=42f9afce22be463b98c3e0bd8296dc18, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.487989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.491192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:03.504965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1207/2400] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:32:03 2022] PUT /identity/v3/projects/1ac4897af78b42bf8dd8edd9be89ff60/users/42f9afce22be463b98c3e0bd8296dc18/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:32:03.510410 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0ef8-c9d8-411c-87b1-233231c6b26b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.510681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0ef8-c9d8-411c-87b1-233231c6b26b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.510950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0ef8-c9d8-411c-87b1-233231c6b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.531055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3b4e0ef8-c9d8-411c-87b1-233231c6b26b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:03.532279 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3b4e0ef8-c9d8-411c-87b1-233231c6b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:03.593629 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1194/2401] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:03.603570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:03.605635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:03.667387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d63f6f-a4b6-4aee-a52c-ca80abc934bf admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:03.671784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:03.672389 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:03.672666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:03.672939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:03.683183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:03.684950 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:03.694575 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1208/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:32:04.399860 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fd8c7-fa01-44d2-98c8-707d67f21bf7 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:04.400444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fd8c7-fa01-44d2-98c8-707d67f21bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:04.400444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fd8c7-fa01-44d2-98c8-707d67f21bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:04.437229 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-fb8fd8c7-fa01-44d2-98c8-707d67f21bf7 None None] MFA Rules not processed for user `42f9afce22be463b98c3e0bd8296dc18`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:04.439007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-fb8fd8c7-fa01-44d2-98c8-707d67f21bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:04.523795 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1195/2403] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:04.534042 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:04.536013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:04.596751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6d067-0625-4eed-bdfc-088d9358393c service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:04.600925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:04.601536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:04.601808 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:04.602074 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:04.610437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:04.612514 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc24da-2c9f-4afb-a8cb-e929b6c66ee9 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:04.621874 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1209/2404] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:32:05.998987 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:05.999585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:05.999585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:06.020773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] MFA Rules not processed for user `42f9afce22be463b98c3e0bd8296dc18`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:06.022030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:06.029273 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1196/2405] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:14.617398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=54 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:16.014604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:21.454892 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7027f-7515-464a-b23e-06a96b1ce290 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:21.455430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7027f-7515-464a-b23e-06a96b1ce290 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:21.455430 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7027f-7515-464a-b23e-06a96b1ce290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:21.487835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-09c7027f-7515-464a-b23e-06a96b1ce290 None None] MFA Rules not processed for user `42f9afce22be463b98c3e0bd8296dc18`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:21.489214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-09c7027f-7515-464a-b23e-06a96b1ce290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:21.516225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1210/2406] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:31.514200 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=15 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:35.780732 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:35.782238 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:35.930717 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6f55d-6af0-418b-b9ea-a4fd412a629d None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:35.936174 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:35.936777 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:35.937051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:35.937315 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:35.939132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:35.950520 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:35.952116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:35.961647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1197/2407] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:32:41.718005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20760a5d-b671-43a0-9445-ec5466cb4b05 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:41.718558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20760a5d-b671-43a0-9445-ec5466cb4b05 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:41.718974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20760a5d-b671-43a0-9445-ec5466cb4b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:41.755921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-20760a5d-b671-43a0-9445-ec5466cb4b05 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:41.757611 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-20760a5d-b671-43a0-9445-ec5466cb4b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:41.828730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1211/2408] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:41.880966 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:41.882514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:41.952532 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace86c1-247d-4013-a510-9764c727a2af service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:41.961043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:41.961842 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:41.962269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:41.962997 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:41.973040 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:41.974754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f208146-c242-4c8b-bb27-29d2ac29638d service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:41.982906 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1198/2409] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:32:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:32:51.821758 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=32 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:51.979590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:32:53.279495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:53.280019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:53.280287 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:53.302653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] MFA Rules not processed for user `75bf2318422245ee8bd06d541e0572fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:53.304283 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:53.366785 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1212/2410] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:57.381373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:57.381876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:57.386406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:57.407656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:57.408904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:32:57.416280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1199/2411] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:32:57.429155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:57.441682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cae421-c4ca-4536-b866-8275ef5946ec None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:57.447375 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:57.447997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:57.448272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:57.448546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:57.457329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:57.458940 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:57.468105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1213/2412] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:32:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:32:59.446135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:59.476381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1c12a-ea21-40ae-9e53-e5392f5263b3 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:59.481012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:59.481715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:59.481996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:59.482263 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] PATH_INFO: `/v3/users/42f9afce22be463b98c3e0bd8296dc18` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:59.483457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f9afce22be463b98c3e0bd8296dc18)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:59.485408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:59.532418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f9afce22be463b98c3e0bd8296dc18'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.544785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f9afce22be463b98c3e0bd8296dc18'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.545450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f9afce22be463b98c3e0bd8296dc18'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.552277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f9afce22be463b98c3e0bd8296dc18'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.552899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-1355a9d2-561d-409e-abd1-a7f3a3c70050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f9afce22be463b98c3e0bd8296dc18'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.567170 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1200/2413] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:32:59 2022] DELETE /identity/v3/users/42f9afce22be463b98c3e0bd8296dc18 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:32:59.753249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:32:59.765414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0789eb-004d-4d3e-869b-5038b1aa65ea service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:32:59.824736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:32:59.825440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:59.825733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:59.826000 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] PATH_INFO: `/v3/projects/1ac4897af78b42bf8dd8edd9be89ff60` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:59.827105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac4897af78b42bf8dd8edd9be89ff60)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:32:59.829043 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:32:59.878045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac4897af78b42bf8dd8edd9be89ff60', 'request_id': 'req-42b9bf56-27ca-43bb-a308-950eeb6b39c3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.884480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac4897af78b42bf8dd8edd9be89ff60', 'request_id': 'req-42b9bf56-27ca-43bb-a308-950eeb6b39c3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.893864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac4897af78b42bf8dd8edd9be89ff60', 'request_id': 'req-42b9bf56-27ca-43bb-a308-950eeb6b39c3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.917342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] The token cache is being invalidate because project 1ac4897af78b42bf8dd8edd9be89ff60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:32:59.917629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.918290 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-42b9bf56-27ca-43bb-a308-950eeb6b39c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:32:59.919873 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1214/2414] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:32:59 2022] DELETE /identity/v3/projects/1ac4897af78b42bf8dd8edd9be89ff60 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:32:59.925773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:32:59.925993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:32:59.926262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:32:59.985863 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:32:59.987254 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:00.048441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1201/2415] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:00.056256 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.056594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.056866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.076548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:00.077761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:00.085703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1215/2416] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:33:00.090482 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.099989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c90be-dbed-4c37-b90b-f49834a6a0e9 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.104509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.105111 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.105384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.105647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.106293 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:33:00.106587 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.108216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.119649 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1202/2417] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:33:00.124132 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.129667 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:00.133439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bdb30-3bf2-46b0-97aa-c35a77489dd1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.145545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.146251 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.146526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.146793 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.147472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.149409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.177689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1216/2418] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:33:00 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:33:00.182025 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.192069 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8124d2-10c8-45d5-be6d-3376bf9fb9e6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.195686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:00.256657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.257266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.257544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.257820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.258454 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.260384 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.287260 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1203/2419] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:33:00 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:33:00.291627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.303029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e1feb-2737-4d19-9bb4-96a93f984ce2 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.308405 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.308997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.309267 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.309531 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.310208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.311851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.322159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1217/2420] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:33:00.326227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.336711 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c8c9-19d5-4ce8-8408-24215efbbc48 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.342771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.343404 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.343684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.343949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.344598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.346228 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf0a91-7a97-488a-a8e1-6745897260ae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.357847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1204/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:33:00.362062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.372685 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63182-74d9-4a48-9a82-bd1221286074 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.378091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.378733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.379014 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.379335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] PATH_INFO: `/v3/projects/b3f7c1cc1b1749d181f5d698d38e439e/users/41d0b8b3e6f34e819876700e51543280/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.394539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f7c1cc1b1749d181f5d698d38e439e, user_id=41d0b8b3e6f34e819876700e51543280, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.397246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.413209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1218/2422] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:33:00 2022] PUT /identity/v3/projects/b3f7c1cc1b1749d181f5d698d38e439e/users/41d0b8b3e6f34e819876700e51543280/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:00.423887 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.424192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.424458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.452745 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] MFA Rules not processed for user `41d0b8b3e6f34e819876700e51543280`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:00.453893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:00.519876 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1205/2423] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:33:00.531593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:00.533675 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:00.617518 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70c93a-f8fb-4132-ab69-f52f7eddc363 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:00.621822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:00.622551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:00.622828 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:00.623101 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:00.632987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:00.634416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954da3b-2f74-4cce-ab09-ebe4a4a6bab6 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:00.643420 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1219/2424] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:33:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:33:03.224880 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:03.290399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4cd50-750e-4fa3-b141-4928e004d1aa None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:03.295074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:03.295819 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:03.296097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:03.296386 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] PATH_INFO: `/v3/users/41d0b8b3e6f34e819876700e51543280` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:03.297490 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d0b8b3e6f34e819876700e51543280)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:03.299519 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:03.316201 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:03.349935 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d0b8b3e6f34e819876700e51543280'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.362084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d0b8b3e6f34e819876700e51543280'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.362786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d0b8b3e6f34e819876700e51543280'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.370414 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d0b8b3e6f34e819876700e51543280'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.371098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d0b8b3e6f34e819876700e51543280'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.385199 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1206/2425] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:33:03 2022] DELETE /identity/v3/users/41d0b8b3e6f34e819876700e51543280 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:03.389917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:03.390467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:03.390467 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:03.409821 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:03.411016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:03.471826 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1220/2426] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:03.481898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:03.483624 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:03.569291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2b07b-5c50-47af-aa88-4be4fe83a602 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:03.573646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:03.574252 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:03.574531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:03.574771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:03.584510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:03.585933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8d5ea-9923-4989-95ed-22aaf593e465 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:03.596312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1207/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:33:03.876690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:03.878329 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:03.891076 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf8d2-eb26-49a3-b6d3-7a73023cfe87 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:03.896455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:03.897097 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:03.897787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:03.897787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] PATH_INFO: `/v3/projects/b3f7c1cc1b1749d181f5d698d38e439e` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:03.898897 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f7c1cc1b1749d181f5d698d38e439e)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:03.900854 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:03.934341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:03.952750 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f7c1cc1b1749d181f5d698d38e439e', 'request_id': 'req-cdd84236-c55b-4131-aaa6-7af0116aaa30'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.959041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f7c1cc1b1749d181f5d698d38e439e', 'request_id': 'req-cdd84236-c55b-4131-aaa6-7af0116aaa30'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.968421 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f7c1cc1b1749d181f5d698d38e439e', 'request_id': 'req-cdd84236-c55b-4131-aaa6-7af0116aaa30'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.979569 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] The token cache is being invalidate because project b3f7c1cc1b1749d181f5d698d38e439e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:33:03.979856 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.980562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-cdd84236-c55b-4131-aaa6-7af0116aaa30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:03.982265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1221/2428] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:33:03 2022] DELETE /identity/v3/projects/b3f7c1cc1b1749d181f5d698d38e439e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:03.987395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:03.987682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:03.987955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.008341 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:04.009556 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:04.077463 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1208/2429] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:04.142377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.142751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.142891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.163932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:04.165138 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:04.174081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1222/2430] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:04.178476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.189187 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94517a-820d-47d0-853e-4b4267eb406b None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.195114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.195771 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.196054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.196318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.196973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:33:04.197277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.198929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.211189 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1209/2431] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:33:04.215915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.225284 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33d480-8a51-4970-9ca6-2c3aa4ce9dcb None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.230416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.231034 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.231297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.231594 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.232292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.234210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.266254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1223/2432] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:33:04 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:33:04.270586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.277101 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:04.281215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408edb-0495-47bc-9efc-53cc9184fe91 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.342437 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.343046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.343365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.343633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.344248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.346181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.372217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1210/2433] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:33:04 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:33:04.376491 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.387194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb91f06-d036-475a-8cc1-d1e587ad8ea4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.392424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.393022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.393292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.393573 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.394207 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.395915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.406684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1224/2434] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:33:04.411558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.422588 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795cbfe-3a27-4ba7-a7f5-fe05ad5cba0a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.428003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.428620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.428893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.429162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.429844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.431522 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891023-a2fc-4189-bce8-d37dbfbae9b1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.442401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1211/2435] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:33:04.446677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.456954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c73fe-9024-4ebf-a882-429595cd42a9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.461790 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.462453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.462752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.463408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] PATH_INFO: `/v3/projects/21054a70b4da44528c776d68e9d044ce/users/ffb55c20677c4e08b53be5e2ab99a818/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.478180 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21054a70b4da44528c776d68e9d044ce, user_id=ffb55c20677c4e08b53be5e2ab99a818, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.480916 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.494796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1225/2436] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:33:04 2022] PUT /identity/v3/projects/21054a70b4da44528c776d68e9d044ce/users/ffb55c20677c4e08b53be5e2ab99a818/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:04.505261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.505547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.505815 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.534622 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] MFA Rules not processed for user `ffb55c20677c4e08b53be5e2ab99a818`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:04.535884 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:04.603171 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1212/2437] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:04.614966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.617023 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:04.676789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9944ad-e43e-423f-b60a-e388ee51e12d admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.681190 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.681796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.682085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.682342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.691119 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.692771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c77a9c-0fd5-4cb0-9ea3-2766cc966521 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.701765 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1226/2438] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:33:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:33:04.728609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.793773 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cd2e0-89f7-4578-8c0a-caec1c5af55c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:04.799110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:04.799770 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.800053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.800330 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] PATH_INFO: `/v3/users/ffb55c20677c4e08b53be5e2ab99a818` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.801524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb55c20677c4e08b53be5e2ab99a818)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:04.803537 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:04.817451 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:04.856491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb55c20677c4e08b53be5e2ab99a818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:04.869975 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb55c20677c4e08b53be5e2ab99a818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:04.870830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb55c20677c4e08b53be5e2ab99a818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:04.878302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb55c20677c4e08b53be5e2ab99a818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:04.878999 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb55c20677c4e08b53be5e2ab99a818'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:04.892875 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1213/2439] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:33:04 2022] DELETE /identity/v3/users/ffb55c20677c4e08b53be5e2ab99a818 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:04.897369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:04.897738 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:04.898108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:04.917963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:04.919625 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:04.970791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:04.984658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1227/2440] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:33:04.996840 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:04.998306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:05.068486 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da1063-dd01-48a0-8ee8-45f79a7758ad admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:05.073669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:05.074285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:05.074561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:05.074826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:05.084861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:05.086320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72818043-fa11-4305-99f7-eb0e7dd74704 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:05.096870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1214/2441] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:33:05.362037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:33:05.363582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:05.376198 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebd796-b0d4-4ce8-91c4-6bf11d96328b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:33:05.381507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:33:05.382264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:05.382563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:05.382802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] PATH_INFO: `/v3/projects/21054a70b4da44528c776d68e9d044ce` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:05.384133 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=21054a70b4da44528c776d68e9d044ce)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:33:05.386078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:33:05.435643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21054a70b4da44528c776d68e9d044ce', 'request_id': 'req-e7f26581-37db-448d-9118-6f9284e25dee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:05.442087 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21054a70b4da44528c776d68e9d044ce', 'request_id': 'req-e7f26581-37db-448d-9118-6f9284e25dee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:05.453469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21054a70b4da44528c776d68e9d044ce', 'request_id': 'req-e7f26581-37db-448d-9118-6f9284e25dee'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:05.464447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] The token cache is being invalidate because project 21054a70b4da44528c776d68e9d044ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:33:05.464735 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:05.465398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:33:05.466924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1228/2442] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:33:05 2022] DELETE /identity/v3/projects/21054a70b4da44528c776d68e9d044ce => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:33:15.092441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=47,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:15.461469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:33:54.507995 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:33:54.508560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:33:54.508560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:33:54.530028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] MFA Rules not processed for user `75bf2318422245ee8bd06d541e0572fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:33:54.531290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:33:54.590380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1215/2443] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:34:04.582360 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=25 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:34:04.865237 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:34:04.866909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:04.931120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f26581-37db-448d-9118-6f9284e25dee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:34:04.935085 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:34:04.935713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:34:04.935989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:34:04.936260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:34:04.938100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:05.022693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:34:05.024314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:34:05.045520 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1229/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:34:15.042719 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=63 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:34:45.073435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:34:45.074894 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:45.140601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cfce-ff8f-45fb-bcda-a7343b9e9058 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:34:45.144681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:34:45.145403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:34:45.145682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:34:45.145933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:34:45.147838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:45.163944 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:34:45.166146 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd7383-5f78-40c0-b9eb-5c6652ce2e49 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:34:45.183656 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1216/2445] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:34:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:34:55.175774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=26 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:34:57.562100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:34:57.563637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:57.578711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b3c6c-5498-48de-b246-999d31309351 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:34:57.585561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:34:57.586144 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:34:57.586383 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:34:57.586649 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:34:57.589402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:34:57.603335 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:34:57.604953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:34:57.628987 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1230/2446] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:35:00.308514 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:35:00.309056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:35:00.309157 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:35:00.328449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] MFA Rules not processed for user `75bf2318422245ee8bd06d541e0572fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:35:00.329653 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:00.337889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1217/2447] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:35:00.670165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:35:00.680212 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e1c03-7c54-4491-be3b-d9569cee4fe8 service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:35:00.684862 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:35:00.685464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:35:00.685742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:35:00.686012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:35:00.688092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:00.698037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:35:00.699495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:35:00.707742 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1231/2448] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:35:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:35:01.035524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:35:01.037061 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:01.112225 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e64b3f-d6e3-4d27-b58d-291ad7f0bd11 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:35:01.118707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:35:01.119349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:35:01.119627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:35:01.119890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:35:01.121897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:01.133286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:35:01.134898 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:35:01.145030 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1218/2449] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:35:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:35:03.240749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:35:03.242371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:03.253299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137564e-8f30-4247-a6bc-3f76173b22fd service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:35:03.257553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:35:03.258278 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:35:03.258620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:35:03.259120 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:35:03.261081 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:03.271802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:35:03.273373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:35:03.282894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1232/2450] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:35:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:35:09.681957 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:35:09.697722 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789050-f8bd-46ea-8524-05379f24a011 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:35:09.703542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:35:09.704607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:35:09.704882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:35:09.705148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:35:09.707020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:35:09.719715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:35:09.721264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:35:09.731927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1219/2451] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:35:13.278554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=18 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:35:19.727696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=41 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:13.055469 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.068668 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec956c2-51bc-4951-92c1-99fbc507efa6 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.075364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.075979 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.076264 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.076678 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.078756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:13.093084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.094761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.106956 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1233/2452] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:36:13.108104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.109627 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:13.180893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8e24e-1f7a-4261-aa73-220c35787c42 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.186008 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.186841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.187068 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.187356 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] PATH_INFO: `/v3/users/75bf2318422245ee8bd06d541e0572fa` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.188504 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bf2318422245ee8bd06d541e0572fa)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.190462 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.242304 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bf2318422245ee8bd06d541e0572fa'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.254286 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bf2318422245ee8bd06d541e0572fa'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.254963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bf2318422245ee8bd06d541e0572fa'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.261841 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bf2318422245ee8bd06d541e0572fa'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.262478 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-9f67339c-758c-4952-87fb-b1c42a9a419a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bf2318422245ee8bd06d541e0572fa'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.276367 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1220/2453] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:36:13 2022] DELETE /identity/v3/users/75bf2318422245ee8bd06d541e0572fa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:13.460635 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.472091 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c06a9-1f84-47a1-b572-a2fc5e407d16 service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.537988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.538616 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.538887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.539172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] PATH_INFO: `/v3/projects/7bedba73ec874058acaa1c78c3312e4b` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.540297 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bedba73ec874058acaa1c78c3312e4b)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.542218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.588010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bedba73ec874058acaa1c78c3312e4b', 'request_id': 'req-ca74b37c-3eec-4281-a9be-20b08b8c4ada'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.594772 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bedba73ec874058acaa1c78c3312e4b', 'request_id': 'req-ca74b37c-3eec-4281-a9be-20b08b8c4ada'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.602360 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bedba73ec874058acaa1c78c3312e4b', 'request_id': 'req-ca74b37c-3eec-4281-a9be-20b08b8c4ada'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.612565 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] The token cache is being invalidate because project 7bedba73ec874058acaa1c78c3312e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:36:13.612847 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.613462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ca74b37c-3eec-4281-a9be-20b08b8c4ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:13.618398 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1234/2454] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:36:13 2022] DELETE /identity/v3/projects/7bedba73ec874058acaa1c78c3312e4b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:13.620083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.620373 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.620674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.641609 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:13.642905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:13.704905 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1221/2455] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:13.713309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.713948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.714215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.734544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:13.735822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:13.743989 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1235/2456] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:13.748434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.758724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9ad3-7a30-49fb-960a-86cd1cd1a277 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.764517 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.765132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.765407 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.765670 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.766319 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:36:13.766619 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.768246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.781137 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1222/2457] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:36:13.786345 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.796062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa346ff4-222f-4c59-b680-3d9605217eb1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.801268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.801954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.802305 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.802643 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.803425 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.805351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.847870 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1236/2458] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:36:13 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:13.849600 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.861193 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad360b1d-ff55-4b9b-8ad3-290b0239b6c7 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.865214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:13.920834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.921481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.921762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.922032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.922666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.924669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.953428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1223/2459] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:36:13 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:13.959537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:13.970960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2801c38-e84b-4f73-8fae-e4d11614fd02 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:13.976155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:13.976881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:13.977161 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:13.977434 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:13.978086 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:13.979781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:13.990331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1237/2460] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:13.994579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.005160 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f5be-e373-4798-ae39-d4cb27e01549 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.010886 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.011602 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.011878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.012144 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.012801 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.014533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c0818-891b-4c59-92d0-bfec583e2202 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.025623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1224/2461] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:14.029966 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.040933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d882c-1572-48bd-9d55-4adaa6e9614e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.047763 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.048540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.048909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.049249 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] PATH_INFO: `/v3/projects/40668b6627504d35b3b20565dab0b619/users/7157d01829b247ff8f3c38667e987ba6/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.065695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40668b6627504d35b3b20565dab0b619, user_id=7157d01829b247ff8f3c38667e987ba6, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.068567 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.084890 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1238/2462] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:14 2022] PUT /identity/v3/projects/40668b6627504d35b3b20565dab0b619/users/7157d01829b247ff8f3c38667e987ba6/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:14.095866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.096130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.096388 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.125231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] MFA Rules not processed for user `7157d01829b247ff8f3c38667e987ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:14.126452 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:14.194122 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1225/2463] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:14.198702 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.204504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:14.263285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d3230-27ea-49a7-bef3-03990a6aea3e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.267961 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.268566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.268846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.269110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.269751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.271692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.302601 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1239/2464] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:36:14 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:14.303592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.314450 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65535370-1e5d-4488-800c-be5420fcae2f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.370203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.371142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.371142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.372106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.372106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.374021 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.396998 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1226/2465] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:36:14 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:14.401416 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.412540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6af980-e31d-4c3a-b4b2-99747c8dbaf0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.418032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.418665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.418971 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.419277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.419929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.421563 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.432214 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1240/2466] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:14.436402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.442951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:14.446821 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307260c-89d2-4f85-a888-57236a001188 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.452387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.453041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.453322 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.453594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] PATH_INFO: `/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7/users/753dc4f4f5d54c8d83ed2c429bd947c6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.470215 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7bfdddeeba4c7d8db9871586081ed7, user_id=753dc4f4f5d54c8d83ed2c429bd947c6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.472899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.488895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1227/2467] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:14 2022] PUT /identity/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7/users/753dc4f4f5d54c8d83ed2c429bd947c6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:14.492904 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.503339 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3c2b-2907-4d09-8f12-711d0f82cfa6 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.561068 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.561628 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.561910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.562209 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.562880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.564546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2da53-d549-4974-9cf0-956efc352e54 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.574535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1241/2468] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:14.578858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.589882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b2f43-8c7c-45dc-ac3c-8edd84e0dca3 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.594938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.595618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.595889 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.596155 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] PATH_INFO: `/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7/users/753dc4f4f5d54c8d83ed2c429bd947c6/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.598212 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7bfdddeeba4c7d8db9871586081ed7, user_id=753dc4f4f5d54c8d83ed2c429bd947c6, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.600858 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.611442 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1228/2469] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:14 2022] PUT /identity/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7/users/753dc4f4f5d54c8d83ed2c429bd947c6/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:14.623315 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.623604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.623880 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.652843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] MFA Rules not processed for user `753dc4f4f5d54c8d83ed2c429bd947c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:14.654059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:14.719408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:14.723185 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1242/2470] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:14.739652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.742560 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:14.802955 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3dcf8-65c0-43c4-b180-ef88bb1da1a6 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.807097 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.807729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.808002 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.808264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.817733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.819080 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:14.828867 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1229/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:36:14.919165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:14.929687 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1588b-4da4-44ce-b946-1dd68dca6a0b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:14.933752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:14.934359 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:14.934637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:14.934911 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:14.997211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:14.999399 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61688e-b730-4022-8cd7-cc80460eb9f7 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:15.011064 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1243/2472] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:36:24.824585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:25.006504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=28 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:54.125461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:54.188945 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb667e6-eede-49b2-acae-1f48fbe4a885 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:54.194247 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:54.194870 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:54.195165 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:54.195413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] PATH_INFO: `/v3/users/7157d01829b247ff8f3c38667e987ba6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:54.196613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7157d01829b247ff8f3c38667e987ba6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:54.198663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:54.246681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7157d01829b247ff8f3c38667e987ba6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.258206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7157d01829b247ff8f3c38667e987ba6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.258872 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7157d01829b247ff8f3c38667e987ba6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.265576 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7157d01829b247ff8f3c38667e987ba6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.266206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7157d01829b247ff8f3c38667e987ba6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.280180 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1230/2473] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:36:54 2022] DELETE /identity/v3/users/7157d01829b247ff8f3c38667e987ba6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:54.284796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:54.285299 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:54.285417 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:54.304954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:54.306149 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:54.374949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1244/2474] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:54.384950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:54.386525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:54.447857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989007-3213-4089-8294-d25491da9e25 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:54.452445 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:54.453037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:54.453309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:54.453590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:54.463901 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:54.465342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:54.475007 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1231/2475] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:36:54.756964 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:54.758454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:54.771682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aae35c-3f62-4980-a1a7-6132b0b09614 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:54.776963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:54.777610 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:54.777877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:54.778150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] PATH_INFO: `/v3/projects/40668b6627504d35b3b20565dab0b619` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:54.779334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=40668b6627504d35b3b20565dab0b619)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:54.781378 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:54.830457 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40668b6627504d35b3b20565dab0b619', 'request_id': 'req-a15332c8-b315-446f-905d-59a23cc52dde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.836789 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40668b6627504d35b3b20565dab0b619', 'request_id': 'req-a15332c8-b315-446f-905d-59a23cc52dde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.846159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40668b6627504d35b3b20565dab0b619', 'request_id': 'req-a15332c8-b315-446f-905d-59a23cc52dde'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.856314 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] The token cache is being invalidate because project 40668b6627504d35b3b20565dab0b619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:36:54.856595 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.857204 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.858752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1245/2476] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:36:54 2022] DELETE /identity/v3/projects/40668b6627504d35b3b20565dab0b619 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:54.862847 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:54.872856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808ea4b-d9c8-4ac2-bd27-f26e8bff0d66 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:54.892686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:54.933394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:54.934028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:54.934307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:54.934585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] PATH_INFO: `/v3/users/753dc4f4f5d54c8d83ed2c429bd947c6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:54.935743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=753dc4f4f5d54c8d83ed2c429bd947c6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:54.937726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:54.986032 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753dc4f4f5d54c8d83ed2c429bd947c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.998616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753dc4f4f5d54c8d83ed2c429bd947c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:54.999411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753dc4f4f5d54c8d83ed2c429bd947c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.006128 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753dc4f4f5d54c8d83ed2c429bd947c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.006724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-3636839c-24f2-4bc0-b031-ff12b2969e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753dc4f4f5d54c8d83ed2c429bd947c6'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.021483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1232/2477] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:36:54 2022] DELETE /identity/v3/users/753dc4f4f5d54c8d83ed2c429bd947c6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:55.299546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.301159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:55.317128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15332c8-b315-446f-905d-59a23cc52dde admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.378289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.378910 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.379175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.379483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] PATH_INFO: `/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.380647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7bfdddeeba4c7d8db9871586081ed7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.382546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.414875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:55.432731 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7bfdddeeba4c7d8db9871586081ed7', 'request_id': 'req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.439033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7bfdddeeba4c7d8db9871586081ed7', 'request_id': 'req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.448931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7bfdddeeba4c7d8db9871586081ed7', 'request_id': 'req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.460730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] The token cache is being invalidate because project ac7bfdddeeba4c7d8db9871586081ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:36:55.461021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.461728 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5cca7d0f-baf1-4f01-85a4-23b4850ef45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:36:55.463717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1246/2478] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:36:55 2022] DELETE /identity/v3/projects/ac7bfdddeeba4c7d8db9871586081ed7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:55.468772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.469248 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.469345 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.489931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:55.491214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:55.553789 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1233/2479] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:55.563277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.563561 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.563830 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.584888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:55.586092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:55.595303 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1247/2480] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:55.600313 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.611364 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b1a3b-9898-4ca6-a591-08de97ef228a None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.616206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.616791 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.617075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.617346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.618065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:36:55.618365 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.620015 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.632938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1234/2481] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:36:55.637544 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.647435 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b4697-9da9-4c6c-9319-bcd4a234d85b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.653145 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.653769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.654026 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.654292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.654952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.656949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.686912 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1248/2482] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:36:55 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:55.691181 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.701355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aeb4a-dcbc-4942-bdda-ced5e750dffb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.727184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:55.762395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.763000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.763310 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.763578 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.764214 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.766175 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.788720 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1235/2483] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:36:55 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:36:55.793195 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.804250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8f80e-a18c-4eec-9b27-6c35c6731f01 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.808867 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.809459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.809730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.809997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.810636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.812620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.823620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1249/2484] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:55.827506 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.837375 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62fb82-7f4d-408e-9974-d1eecd856f91 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.842640 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.843297 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.843570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.843835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.844470 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.846059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a12a-fe85-46e0-95c3-0183bea33984 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.856845 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1236/2485] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:55.861233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:55.872943 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd63519-c5b0-4df3-99c4-e961d0564445 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:55.879619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:55.880286 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.880562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.880836 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] PATH_INFO: `/v3/projects/0e8e434022534561b684346c036e669f/users/59def942ab79493c9ed50b19f4f65901/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:55.898061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8e434022534561b684346c036e669f, user_id=59def942ab79493c9ed50b19f4f65901, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:55.900781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:55.919933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1250/2486] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:55 2022] PUT /identity/v3/projects/0e8e434022534561b684346c036e669f/users/59def942ab79493c9ed50b19f4f65901/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:55.986355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:55.986655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:55.986900 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.016996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] MFA Rules not processed for user `59def942ab79493c9ed50b19f4f65901`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:56.018178 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:56.084964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1237/2487] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:56.090226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.157917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c92572-5891-42aa-a6e7-69e85d35f148 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.163464 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.164070 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.164347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.164613 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.165210 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.167128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.200984 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1251/2488] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:36:56 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:56.205326 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.216647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79fa98-e506-4901-a076-60ef9605fbcb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.274993 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.275574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.275855 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.276123 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.276785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.278879 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.302607 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1238/2489] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:36:56 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:36:56.306781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.317368 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e41eeb-505b-4a39-9c2c-ded8abf3ab34 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.322401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.323012 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.323336 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.323618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.324270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.325933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.337072 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1252/2490] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:36:56.341547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.351830 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80817ec-783b-4145-a4f1-964b6519c7f2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.356907 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.357552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.357826 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.358091 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] PATH_INFO: `/v3/projects/46bcdb00187d4b5eb34082482b967985/users/86fca195025846a6818cfb2c0c24935a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.374747 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46bcdb00187d4b5eb34082482b967985, user_id=86fca195025846a6818cfb2c0c24935a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.377417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.393816 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1239/2491] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:56 2022] PUT /identity/v3/projects/46bcdb00187d4b5eb34082482b967985/users/86fca195025846a6818cfb2c0c24935a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:56.398556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.409082 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae5a64-c972-4502-8584-1f77bfb4e54b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.430554 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:56.470337 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.470949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.471258 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.471535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.472189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.473778 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07812172-fad9-46f3-8f18-0b6d49b06617 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.483454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1253/2492] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:36:56.488044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.496300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:36:56.503467 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe46f1-0aee-40dc-9447-199809e9a074 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.509402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.510093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.510487 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.510764 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] PATH_INFO: `/v3/projects/46bcdb00187d4b5eb34082482b967985/users/86fca195025846a6818cfb2c0c24935a/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.512892 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46bcdb00187d4b5eb34082482b967985, user_id=86fca195025846a6818cfb2c0c24935a, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.515598 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.527093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1240/2493] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:36:56 2022] PUT /identity/v3/projects/46bcdb00187d4b5eb34082482b967985/users/86fca195025846a6818cfb2c0c24935a/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:36:56.537582 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.537878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.538154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.567545 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] MFA Rules not processed for user `86fca195025846a6818cfb2c0c24935a`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:36:56.568755 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:56.645059 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1254/2494] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:36:56.656929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.658703 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:56.718235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685375-b2c1-4cab-9d4a-28899bec195a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.722690 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.723323 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.723604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.723869 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.733483 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.734916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.744854 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1241/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:36:56.793631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:36:56.795224 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:36:56.808442 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb8445-dda4-4f58-a169-36e3d33236f1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:36:56.813080 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:36:56.813677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:36:56.813952 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:36:56.814223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:36:56.877507 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:36:56.878948 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fe8bf-e2c2-418d-b200-e6ede081a412 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:36:56.887953 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1255/2496] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:00.266358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:00.331014 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9635ede-bf4e-451e-a54f-23d85f39d771 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:00.335712 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:00.336342 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:00.336620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:00.336896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] PATH_INFO: `/v3/users/59def942ab79493c9ed50b19f4f65901` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:00.338402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59def942ab79493c9ed50b19f4f65901)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:00.340371 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:00.389705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59def942ab79493c9ed50b19f4f65901'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.401753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59def942ab79493c9ed50b19f4f65901'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.402554 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59def942ab79493c9ed50b19f4f65901'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.409793 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59def942ab79493c9ed50b19f4f65901'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.410436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59def942ab79493c9ed50b19f4f65901'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.424340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1242/2497] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:00 2022] DELETE /identity/v3/users/59def942ab79493c9ed50b19f4f65901 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:00.428859 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:00.429372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:00.429489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:00.449371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:00.450591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:00.492915 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=100 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:00.513458 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1256/2498] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:00.524268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:00.526162 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:00.540754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:00.590418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d399567-653f-4c89-9864-75e5079aa2f4 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:00.594883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:00.595539 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:00.595818 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:00.596090 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:00.605547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:00.606967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:00.616787 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1243/2499] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:00.894272 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:00.895783 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:00.908925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfab85-f17a-48a4-b62c-bfda0edd9693 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:00.914238 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:00.914893 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:00.915197 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:00.915475 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] PATH_INFO: `/v3/projects/0e8e434022534561b684346c036e669f` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:00.916631 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8e434022534561b684346c036e669f)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:00.918636 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:00.970515 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8e434022534561b684346c036e669f', 'request_id': 'req-27c9133f-1319-4ea9-b6cf-980831cf9aac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.977174 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8e434022534561b684346c036e669f', 'request_id': 'req-27c9133f-1319-4ea9-b6cf-980831cf9aac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.987777 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8e434022534561b684346c036e669f', 'request_id': 'req-27c9133f-1319-4ea9-b6cf-980831cf9aac'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.998556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] The token cache is being invalidate because project 0e8e434022534561b684346c036e669f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:00.998864 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:00.999602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.001114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1257/2500] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:00 2022] DELETE /identity/v3/projects/0e8e434022534561b684346c036e669f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:01.005529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.015261 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796be1e-5ba8-4944-aea2-89856026552b service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.073124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.073746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.074020 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.074272 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] PATH_INFO: `/v3/users/86fca195025846a6818cfb2c0c24935a` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.075468 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fca195025846a6818cfb2c0c24935a)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.077417 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.131312 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fca195025846a6818cfb2c0c24935a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.144772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fca195025846a6818cfb2c0c24935a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.145465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fca195025846a6818cfb2c0c24935a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.152685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fca195025846a6818cfb2c0c24935a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.154354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b98b747e-2d3d-4671-9fb6-6b6fe51bcc90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fca195025846a6818cfb2c0c24935a'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.172774 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1244/2501] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:01 2022] DELETE /identity/v3/users/86fca195025846a6818cfb2c0c24935a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:01.448168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.449638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:01.465021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9133f-1319-4ea9-b6cf-980831cf9aac admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.525550 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.526281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.526558 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.526852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] PATH_INFO: `/v3/projects/46bcdb00187d4b5eb34082482b967985` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.528084 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=46bcdb00187d4b5eb34082482b967985)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.529995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.576931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46bcdb00187d4b5eb34082482b967985', 'request_id': 'req-a00d432e-3430-490a-985d-8b7b99af1e8a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.583208 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46bcdb00187d4b5eb34082482b967985', 'request_id': 'req-a00d432e-3430-490a-985d-8b7b99af1e8a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.592539 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46bcdb00187d4b5eb34082482b967985', 'request_id': 'req-a00d432e-3430-490a-985d-8b7b99af1e8a'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.598959 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:01.604093 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] The token cache is being invalidate because project 46bcdb00187d4b5eb34082482b967985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:01.604364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.604986 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-a00d432e-3430-490a-985d-8b7b99af1e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:01.606894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1258/2502] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:01 2022] DELETE /identity/v3/projects/46bcdb00187d4b5eb34082482b967985 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:37:01.611921 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.612399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.612498 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.633132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:01.634408 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:01.666726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:01.699760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1245/2503] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:01.709277 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.709593 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.709866 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.730013 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:01.731217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:01.739872 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1259/2504] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:01.744471 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.754449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6295608-ef21-46a0-aa77-741c6733285f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.759479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.760205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.760472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.760746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.761400 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:37:01.761696 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.763406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.775973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1246/2505] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 25 13:37:01.782226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.792016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bd516-514d-467c-aa1f-1cc91da2b26d None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.797996 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.798603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.798885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.799196 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.799824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.801773 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.833139 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1260/2506] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:01 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:01.838421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.849245 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e9bba-8170-495f-b5ac-9129759fb809 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.911456 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.912041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.912302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.912570 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.913209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.915177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.939029 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1247/2507] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:01 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:01.944488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.955462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ccd23-83f4-4105-9f6e-a751f98ec87a admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.961016 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.961622 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.961899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.962165 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.963192 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.964857 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:01.975226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1261/2508] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:01.979300 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:01.989589 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6874-eae7-4889-b0bc-2dfabc2024ab admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:01.994883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:01.995544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:01.995822 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:01.996094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:01.996740 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:01.998325 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa1407-f2ad-4be6-9eb5-deea3bd2c8de admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.009082 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1248/2509] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:02.013379 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.023896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c176d69-11e2-4978-86d8-89b56befb7d0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.028769 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.029427 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.029701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.029970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] PATH_INFO: `/v3/projects/284ffa6a45c84ea0b8d816015ff9fef9/users/72981dba755843179ad013c41325d4a8/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.045078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284ffa6a45c84ea0b8d816015ff9fef9, user_id=72981dba755843179ad013c41325d4a8, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.047774 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.062412 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1262/2510] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:02 2022] PUT /identity/v3/projects/284ffa6a45c84ea0b8d816015ff9fef9/users/72981dba755843179ad013c41325d4a8/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:37:02.072754 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.073042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.073316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.104059 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] MFA Rules not processed for user `72981dba755843179ad013c41325d4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:02.105266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:02.176434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1249/2511] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:02.181254 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.249493 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d23eec-541f-4152-a54f-a5e0d4873686 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.255492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.256100 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.256372 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.256638 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.257260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.259137 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.289194 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1263/2512] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:02 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:02.293590 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.304488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31916-5f88-449c-a861-8de06cbaae0c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.330752 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:02.366368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.366971 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.367285 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.367549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.368176 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.370093 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.392045 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1250/2513] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:02 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:02.396324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.407030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3e3c1-b0cf-40c8-9003-188c1666801b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.412323 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.412990 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.413268 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.413541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.414183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.415851 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.426619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1264/2514] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:02.430914 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.441604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b89fb-2ab3-4e60-bc19-76ce419e6d7d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.446697 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.447381 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.447654 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.447923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] PATH_INFO: `/v3/projects/75f214bec15046e392f4eb45d249edf6/users/8b093a919e274d66ba852e86c5dea421/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.463083 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f214bec15046e392f4eb45d249edf6, user_id=8b093a919e274d66ba852e86c5dea421, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.465844 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.481270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1251/2515] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:02 2022] PUT /identity/v3/projects/75f214bec15046e392f4eb45d249edf6/users/8b093a919e274d66ba852e86c5dea421/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 25 13:37:02.485548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.495875 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66940ef-f7fd-4891-8d36-834bbff7ac9b admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.552816 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.553343 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.553618 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.553885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.554526 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.556222 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7011a-08ab-4007-83db-69ca8588b67e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.566409 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1265/2516] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:37:02.570857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.582124 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644a700-9d40-4c57-be63-0f0c2cf62a73 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.588394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.589078 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.589351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.589616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] PATH_INFO: `/v3/projects/75f214bec15046e392f4eb45d249edf6/users/8b093a919e274d66ba852e86c5dea421/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.591814 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f214bec15046e392f4eb45d249edf6, user_id=8b093a919e274d66ba852e86c5dea421, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.594476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.605159 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1252/2517] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:02 2022] PUT /identity/v3/projects/75f214bec15046e392f4eb45d249edf6/users/8b093a919e274d66ba852e86c5dea421/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:02.618455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.618935 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.619391 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.628560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:02.651574 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] MFA Rules not processed for user `8b093a919e274d66ba852e86c5dea421`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:02.652800 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:02.721260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1266/2518] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:02.733469 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.735334 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:02.796436 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365778-10f8-472d-a86b-6ebbb67b5d13 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.800583 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.801184 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.801461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.801733 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:02.811317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:02.812685 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:02.822702 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1253/2519] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:02.930055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:02.940396 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6d95c-c049-4c81-a05e-97a5b928a8b1 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:02.944659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:02.945273 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:02.945540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:02.945825 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:03.005220 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:03.006677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cc9ce-3808-4d44-9c32-67309784264b service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:03.016634 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1267/2520] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:03.290481 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:03.363098 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a3a1-cede-4eec-8d2d-e40dc3b49ece service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:03.368701 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:03.369340 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:03.369621 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:03.369960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] PATH_INFO: `/v3/users/72981dba755843179ad013c41325d4a8` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:03.371076 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72981dba755843179ad013c41325d4a8)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:03.373051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:03.407476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:03.428129 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72981dba755843179ad013c41325d4a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:03.439857 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72981dba755843179ad013c41325d4a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:03.440513 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72981dba755843179ad013c41325d4a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:03.447331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72981dba755843179ad013c41325d4a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:03.447996 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72981dba755843179ad013c41325d4a8'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:03.462235 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1254/2521] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:03 2022] DELETE /identity/v3/users/72981dba755843179ad013c41325d4a8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:03.467235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:03.467614 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:03.467887 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:03.487683 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:03.488919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:03.557974 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1268/2522] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:03.567937 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:03.569676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:03.632428 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bca3-2b03-48a4-9aaf-70aec0a23951 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:03.636950 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:03.637634 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:03.638218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:03.638355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:03.650594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:03.652092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:03.662217 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1255/2523] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:03.931407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:03.932994 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:03.946250 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25867a-b50b-4264-a68a-6d70e2a32d42 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:03.951710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:03.952369 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:03.952641 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:03.952903 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] PATH_INFO: `/v3/projects/284ffa6a45c84ea0b8d816015ff9fef9` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:03.954676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=284ffa6a45c84ea0b8d816015ff9fef9)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:03.956927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:03.982967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:04.007771 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284ffa6a45c84ea0b8d816015ff9fef9', 'request_id': 'req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.014009 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284ffa6a45c84ea0b8d816015ff9fef9', 'request_id': 'req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.022302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284ffa6a45c84ea0b8d816015ff9fef9', 'request_id': 'req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.032655 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] The token cache is being invalidate because project 284ffa6a45c84ea0b8d816015ff9fef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:04.032938 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.033556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.035030 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1269/2524] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:03 2022] DELETE /identity/v3/projects/284ffa6a45c84ea0b8d816015ff9fef9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:04.039307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.049309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e08820-a4da-40f5-b28f-80ea0ef5c862 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:04.113510 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:04.114307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.114614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.114902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] PATH_INFO: `/v3/users/8b093a919e274d66ba852e86c5dea421` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.116418 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b093a919e274d66ba852e86c5dea421)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:04.118402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:04.166413 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b093a919e274d66ba852e86c5dea421'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.167594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:04.181317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b093a919e274d66ba852e86c5dea421'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.181908 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b093a919e274d66ba852e86c5dea421'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.188666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b093a919e274d66ba852e86c5dea421'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.189269 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-29f1e35c-9209-48c2-9245-973ed1914f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b093a919e274d66ba852e86c5dea421'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.203663 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1256/2525] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:04 2022] DELETE /identity/v3/users/8b093a919e274d66ba852e86c5dea421 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:04.498999 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.503347 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:04.517373 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dfdd1-2c31-44fa-91e9-b473fa52c2d9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:04.576824 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:04.577455 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.577733 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.577998 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] PATH_INFO: `/v3/projects/75f214bec15046e392f4eb45d249edf6` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.579169 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f214bec15046e392f4eb45d249edf6)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:04.581155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:04.629533 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f214bec15046e392f4eb45d249edf6', 'request_id': 'req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.635965 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f214bec15046e392f4eb45d249edf6', 'request_id': 'req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.646914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f214bec15046e392f4eb45d249edf6', 'request_id': 'req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.658279 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] The token cache is being invalidate because project 75f214bec15046e392f4eb45d249edf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:04.658652 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.659334 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-b6c4eefb-ef43-4f1d-98ac-e1047f05878b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:04.660848 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1270/2526] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:04 2022] DELETE /identity/v3/projects/75f214bec15046e392f4eb45d249edf6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:04.665911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.666262 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.666531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.686585 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:04.687803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:04.749035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1257/2527] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 25 13:37:04.757489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.757881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.758159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.778908 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:04.780159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:04.789566 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1271/2528] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:04.793861 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.803838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508a7a0-8e53-4745-8ac4-65c33d86ef3f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:04.808932 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:04.809542 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.809953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.810232 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.810895 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:37:04.811224 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:04.812866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:04.827126 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1258/2529] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:37:04.831541 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.841692 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2acf-916c-45d7-b1a2-01787c6eff18 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:04.847591 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:04.848225 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.848498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.848764 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.849422 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:04.851377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:04.882384 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1272/2530] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:04 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:04.886699 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.896949 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede696-6f8f-4c72-b42a-12be9087463f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:04.954695 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:04.955314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:04.955592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:04.955852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:04.956489 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:04.958402 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:04.982043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1259/2531] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:04 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:04.986265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:04.997166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde7c0-cd11-4d58-941f-81d3389c75eb admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.002276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.002806 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.003053 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.003348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.004029 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.005620 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.015623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1273/2532] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:05.019626 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.029807 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a95714-6bd8-4b5b-b044-cd765dc7ba70 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.034728 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.035362 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.035646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.035919 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.036568 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.038188 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d4870-c2f1-4f64-a756-8df109921f8f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.048953 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1260/2533] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:05.053094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.058489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:05.064807 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262ec7-f7de-4596-bc24-d78afbee873c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.070846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.071551 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.071834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.072108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] PATH_INFO: `/v3/projects/238e56b814604a4d9054b9c80e3cb492/users/bd136ec2c72c4526898750f8bb0106e2/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.087885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] RBAC: Authorizing `identity:create_grant(project_id=238e56b814604a4d9054b9c80e3cb492, user_id=bd136ec2c72c4526898750f8bb0106e2, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.090577 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.104529 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1274/2534] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:05 2022] PUT /identity/v3/projects/238e56b814604a4d9054b9c80e3cb492/users/bd136ec2c72c4526898750f8bb0106e2/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:05.116056 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.116346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.116618 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.146275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] MFA Rules not processed for user `bd136ec2c72c4526898750f8bb0106e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:05.147552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:05.166652 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:05.215243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1261/2535] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:05.219929 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.282822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd46815-d1b3-40eb-b9cd-4a3f94a4185c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.287553 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.288162 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.288443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.288717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.289341 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.291310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.322392 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1275/2536] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:05 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:05.326647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.338028 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4995f-2dea-4125-b9ea-332652953b76 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.395688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.396302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.396581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.396852 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.397488 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.399458 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.422756 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1262/2537] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:05 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:05.427121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.437751 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d3e4-8933-485d-86a5-c21d85def4b9 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.442840 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.443488 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.443780 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.444039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.444682 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.446281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.458462 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1276/2538] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:05.463062 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.473535 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af646b-417a-47a6-8a91-8b1b4d9730fb admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.479324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.480003 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.480278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.480541 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] PATH_INFO: `/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66/users/512398ff543345678d48a4b4e4451555/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.496120 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ff7ec989d247268b7ae5bdd60c0e66, user_id=512398ff543345678d48a4b4e4451555, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.498769 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.514089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1263/2539] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:05 2022] PUT /identity/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66/users/512398ff543345678d48a4b4e4451555/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:05.518150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.528437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512e3dd-bb1e-4a41-9e74-fa11d6ad5f16 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.586092 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.586646 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.586919 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.587193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.588233 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.589846 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357f73f-730f-4118-b8df-048d7303a264 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.599879 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1277/2540] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:05.604148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.614380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c645-1535-4cfd-b049-362d91abacba admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.619395 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.620050 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.620331 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.620592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] PATH_INFO: `/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66/users/512398ff543345678d48a4b4e4451555/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.622601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ff7ec989d247268b7ae5bdd60c0e66, user_id=512398ff543345678d48a4b4e4451555, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.625271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.635922 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1264/2541] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:05 2022] PUT /identity/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66/users/512398ff543345678d48a4b4e4451555/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:05.647041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.647362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.647630 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.676447 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] MFA Rules not processed for user `512398ff543345678d48a4b4e4451555`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:05.677756 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:05.717962 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:05.750327 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1278/2542] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:05.761646 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:05.763534 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:05.821270 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7cb0-11c7-4880-aec9-50c119ed3bae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:05.825545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:05.826148 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:05.826421 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:05.826688 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:05.856194 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:05.888281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:05.889729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:05.898911 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1265/2543] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:10.202463 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:10.203963 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:10.216727 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32398d83-ee79-4721-8cd1-9281ede656cc None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:10.220624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:10.221221 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:10.221562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:10.221822 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:10.230985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:10.232440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cea30b-ddf3-4f77-9248-ab3792bafb66 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:10.242116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1279/2544] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:15.169333 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:15.233896 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651519aa-a747-4e5f-a7ee-6d9c2f2822e4 service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:15.238676 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:15.239328 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:15.239605 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:15.239877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] PATH_INFO: `/v3/users/bd136ec2c72c4526898750f8bb0106e2` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:15.241037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd136ec2c72c4526898750f8bb0106e2)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:15.242961 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:15.292010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd136ec2c72c4526898750f8bb0106e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.304726 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd136ec2c72c4526898750f8bb0106e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.305431 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd136ec2c72c4526898750f8bb0106e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.312762 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd136ec2c72c4526898750f8bb0106e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.313425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd136ec2c72c4526898750f8bb0106e2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.327719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1266/2545] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:15 2022] DELETE /identity/v3/users/bd136ec2c72c4526898750f8bb0106e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:15.332377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:15.332907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:15.332907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:15.352710 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:15.353889 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:15.414296 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1280/2546] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:15.426132 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:15.428444 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:15.488959 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:15.495084 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c9b80-4da3-4291-bb76-d180c642579d admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:15.499707 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:15.500610 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:15.500929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:15.501349 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:15.511004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:15.512466 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:15.522094 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1267/2547] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:15.792933 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:15.794459 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:15.806835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d60bf-c9dc-47c4-ac52-8749af3554a9 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:15.811852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:15.812492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:15.812770 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:15.813039 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] PATH_INFO: `/v3/projects/238e56b814604a4d9054b9c80e3cb492` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:15.814228 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238e56b814604a4d9054b9c80e3cb492)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:15.816270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:15.865110 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238e56b814604a4d9054b9c80e3cb492', 'request_id': 'req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.871523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238e56b814604a4d9054b9c80e3cb492', 'request_id': 'req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.880605 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238e56b814604a4d9054b9c80e3cb492', 'request_id': 'req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.891410 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] The token cache is being invalidate because project 238e56b814604a4d9054b9c80e3cb492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:15.891697 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.892351 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:15.893930 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1281/2548] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:15 2022] DELETE /identity/v3/projects/238e56b814604a4d9054b9c80e3cb492 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:15.898043 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:15.907669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d976c75-a387-4223-8987-aee52daba80d service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:15.966746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:15.967377 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:15.967657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:15.967924 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] PATH_INFO: `/v3/users/512398ff543345678d48a4b4e4451555` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:15.969292 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512398ff543345678d48a4b4e4451555)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:15.971346 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.034158 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512398ff543345678d48a4b4e4451555'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.048067 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512398ff543345678d48a4b4e4451555'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.048850 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512398ff543345678d48a4b4e4451555'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.056268 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512398ff543345678d48a4b4e4451555'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.056904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b436daff-b725-4d40-8c72-40b67b2b2c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512398ff543345678d48a4b4e4451555'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.070718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1268/2549] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:37:15 2022] DELETE /identity/v3/users/512398ff543345678d48a4b4e4451555 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:16.366231 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.367761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:16.385842 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b117f6-6f21-44ad-a60e-a1382c9cbbf0 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.430761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:16.454201 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.455131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.455570 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.455957 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] PATH_INFO: `/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.457730 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ff7ec989d247268b7ae5bdd60c0e66)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.459747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.517094 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ff7ec989d247268b7ae5bdd60c0e66', 'request_id': 'req-dec72482-c102-4b3e-b77f-e1e245ab3d21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.524977 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ff7ec989d247268b7ae5bdd60c0e66', 'request_id': 'req-dec72482-c102-4b3e-b77f-e1e245ab3d21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.534657 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ff7ec989d247268b7ae5bdd60c0e66', 'request_id': 'req-dec72482-c102-4b3e-b77f-e1e245ab3d21'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.545586 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] The token cache is being invalidate because project f5ff7ec989d247268b7ae5bdd60c0e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81436) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:37:16.545868 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.546492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-dec72482-c102-4b3e-b77f-e1e245ab3d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:37:16.548022 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1282/2550] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:37:16 2022] DELETE /identity/v3/projects/f5ff7ec989d247268b7ae5bdd60c0e66 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:16.554368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.554888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.554888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.575142 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:16.576379 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:16.641010 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1269/2551] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:16.649623 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.649899 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.650154 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.670041 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:16.671262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:16.679597 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1283/2552] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:16.683893 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.694223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed951d7f-65f6-482b-a615-68e16d69e0bb None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.699951 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.700555 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.700834 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.701114 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] PATH_INFO: `/v3/domains` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.701743 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81437) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:37:16.702041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.703686 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.706085 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:16.718012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1270/2553] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:37:16.721293 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.730896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca6206-2458-423e-a2e8-28b5509e2a3f None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.737011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.737612 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.737888 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.738152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.738786 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.740762 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.771433 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1284/2554] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:16.775561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.789230 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323578f-fd56-4f53-96a1-e1efa11b8fda admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.849476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.850075 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.850351 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.850616 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.851274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.853239 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.875662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1271/2555] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:16 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:16.880037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.890407 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbc0c4-c3d4-4584-9978-51b59b861aac admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.895167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.895796 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.896056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.896324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.896967 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.898556 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.908540 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1285/2556] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:16.912859 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.927862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4491a0-55af-47a7-be25-5056d0ba23dd admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.933405 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.934005 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.934291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.934563 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.935246 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.936881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c2fb8-7638-4579-b25a-e4b7d6096609 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:16.947647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1272/2557] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:37:16.952061 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:16.962918 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af459c5-96c4-4656-a578-4254e2ec56f7 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:16.968056 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:16.968711 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:16.968988 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:16.969260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] PATH_INFO: `/v3/projects/91560178b31a49a09ae7926966d0b77f/users/9db67b77b37b4c228499044749eeeaa3/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:16.984010 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91560178b31a49a09ae7926966d0b77f, user_id=9db67b77b37b4c228499044749eeeaa3, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:16.986699 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:17.001408 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1286/2558] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:16 2022] PUT /identity/v3/projects/91560178b31a49a09ae7926966d0b77f/users/9db67b77b37b4c228499044749eeeaa3/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:17.006403 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca422-e908-4d87-bbf8-6ea57c68e7e3 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:17.006705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca422-e908-4d87-bbf8-6ea57c68e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:17.007001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ca422-e908-4d87-bbf8-6ea57c68e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:17.026635 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-788ca422-e908-4d87-bbf8-6ea57c68e7e3 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:17.027827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-788ca422-e908-4d87-bbf8-6ea57c68e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:17.089164 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1273/2559] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:17.101782 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:17.104681 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:17.168218 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b9c03-b838-4f4c-a562-e040ef1a1988 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:17.172891 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:17.173499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:17.173757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:17.174015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:17.184677 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:17.186111 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:17.195997 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1287/2560] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:21.126473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.127009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.127009 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.159933 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] MFA Rules not processed for user `9db67b77b37b4c228499044749eeeaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:21.161302 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:21.227947 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1274/2561] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:21.232559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.242939 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4ecb9-4027-4658-b79b-02980b7b2bec service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.247803 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.248413 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.248689 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.248954 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.249589 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.251535 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.263659 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:21.284736 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1288/2562] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:37:21 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:37:21.285990 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.293491 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:21.298760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f7857-2dcf-4c28-b6f3-d9e2fb018986 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.364713 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.365332 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.365614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.365888 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] PATH_INFO: `/v3/users` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.366531 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.368479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.394584 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1275/2563] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:37:21 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 25 13:37:21.396055 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.407358 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9caf0-723d-449b-92c2-f878ae233c5e admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.414907 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.416523 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.416805 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.417077 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.421801 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.423498 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.435020 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1289/2564] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:37:21.439544 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.450617 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6679b-4606-4248-aaf5-a9df46304468 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.456339 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.456974 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.457443 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.457783 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] PATH_INFO: `/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6/users/9cba3e0d96744847b81eb60565541982/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.475434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a4289ba87f4e0f8e7f8959b01cf7f6, user_id=9cba3e0d96744847b81eb60565541982, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.478321 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.492249 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1276/2565] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:21 2022] PUT /identity/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6/users/9cba3e0d96744847b81eb60565541982/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:21.496270 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.507583 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e4f8-2c1d-4e5b-8ee3-5f821c2cd9ad admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.570381 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.571002 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.571310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.571584 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.572235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.573898 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dce9d-40d7-4f70-8db1-503e02ee6c58 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.584477 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1290/2566] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:37:21.589856 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:21.600832 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050b5f8-f51a-4bd8-9c2e-c69e6184c46f admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:21.606038 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:21.606681 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:21.606958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:21.607220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] PATH_INFO: `/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6/users/9cba3e0d96744847b81eb60565541982/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:21.609209 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a4289ba87f4e0f8e7f8959b01cf7f6, user_id=9cba3e0d96744847b81eb60565541982, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:21.611885 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:21.622915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1277/2567] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:37:21 2022] PUT /identity/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6/users/9cba3e0d96744847b81eb60565541982/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:37:26.293437 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b736-bfb8-402a-9f28-6e3e3ced91df None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:26.294005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b736-bfb8-402a-9f28-6e3e3ced91df None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:26.294005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907b736-bfb8-402a-9f28-6e3e3ced91df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:26.333445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-e907b736-bfb8-402a-9f28-6e3e3ced91df None None] MFA Rules not processed for user `9cba3e0d96744847b81eb60565541982`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:26.334693 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-e907b736-bfb8-402a-9f28-6e3e3ced91df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:26.419510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1291/2568] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:26.433107 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:26.435037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:26.515835 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b124-e240-4efa-a3af-0c597d79515a admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:26.519798 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e49839dc5bc47d6a9a91800fef249c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:26.520406 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:26.520691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:26.520934 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:26.571438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:26.580772 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:26.583613 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:26.593866 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1278/2569] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:26.733639 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623d326-1633-4ef7-b8aa-7b50a4a86e0d None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:26.734015 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623d326-1633-4ef7-b8aa-7b50a4a86e0d None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:26.734472 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623d326-1633-4ef7-b8aa-7b50a4a86e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:26.786131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f623d326-1633-4ef7-b8aa-7b50a4a86e0d None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:26.787376 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f623d326-1633-4ef7-b8aa-7b50a4a86e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:26.821235 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:26.855927 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1292/2570] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:26.905545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:26.907037 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:27.001306 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f54002-c796-4498-a7c0-e2d2d4998896 service glance] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:27.007092 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:27.007729 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:27.007989 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:27.008271 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:27.016940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:27.018524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:27.026909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1279/2571] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:37:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:37:35.630878 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:35.631445 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:35.631559 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:35.651320 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] MFA Rules not processed for user `9db67b77b37b4c228499044749eeeaa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:35.652530 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:35.660021 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1293/2572] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:35.929197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:35.931432 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:36.001046 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce642-6bcf-4d84-8ebf-f560586fbb22 service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:36.005628 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:36.006243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:36.006515 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:36.006782 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:36.008940 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:36.018899 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:36.020352 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc998c-fe80-46f0-a69f-cd254b14fdb5 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:36.028425 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1280/2573] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:36.914572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:36.916125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:36.987128 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256963ae-9dec-4d59-9821-5e2b88d14b3b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:36.993199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:36.993852 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:36.994131 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:36.994401 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:36.996503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:37.008528 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:37.010116 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:37.020211 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1294/2574] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:37:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:37:38.285074 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:38.285549 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:38.285657 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:38.305929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:37:38.306543 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:37:38.314203 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1281/2575] 10.222.0.36 () {58 vars in 1009 bytes} [Tue Oct 25 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:37:38.327071 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:37:38.340260 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328ad3e-29fa-459d-bd76-193b4648f25c service placement] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:37:38.346019 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:37:38.346709 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:37:38.347470 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:37:38.347752 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:37:38.356440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:37:38.358155 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:37:38.367362 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1295/2576] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:37:48.300086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=74 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:37:48.363829 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=53 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:38:06.900581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:38:06.910958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07403c-0587-4028-959b-20da2408ae1c None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:38:06.915368 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:38:06.916004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:38:06.916283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:38:06.916586 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:38:06.926177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:38:06.927641 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:38:06.937110 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1282/2577] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:38:08.759861 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:38:08.762321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:38:08.849695 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7197c-950d-46ee-9d9a-16ca39f56570 service swift] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:38:08.856787 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:38:08.857414 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:38:08.857834 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:38:08.858106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:38:08.860308 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:38:08.874078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:38:08.875924 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fea572-95ed-4171-911a-94e6c577a44a service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:38:08.895757 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1296/2578] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:38:16.933227 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:38:18.883282 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=34 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:21.881727 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:21.883429 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:21.940927 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16be549-c756-48b4-9c2f-4b61cbd637c6 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:21.945205 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:21.945820 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:21.946119 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:21.946358 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:21.948130 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:21.963734 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:21.965216 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:21.976691 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1283/2579] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:39:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:39:26.941032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:26.941517 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:26.941619 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:26.962973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] MFA Rules not processed for user `5afba0fee3f74a5883257487f6c5c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:26.964187 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:26.973152 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1297/2580] 10.222.0.36 () {58 vars in 967 bytes} [Tue Oct 25 13:39:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:27.015324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:27.027320 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2178b6-7385-4f25-b661-171846d7648f service cinder] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:27.033441 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '084038e0639b49c99485f53463392d11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:27.034044 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:27.034317 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:27.034581 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:27.043065 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:27.044718 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:27.052569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1284/2581] 10.222.0.36 () {60 vars in 1384 bytes} [Tue Oct 25 13:39:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 25 13:39:33.566945 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:33.638271 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b71f0c-9dd4-4fb1-a9f7-9cd9e047a36b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:33.642932 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:33.643658 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:33.643934 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:33.644262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] PATH_INFO: `/v3/users/9db67b77b37b4c228499044749eeeaa3` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:33.645426 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db67b77b37b4c228499044749eeeaa3)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:33.647443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:33.697603 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db67b77b37b4c228499044749eeeaa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:33.711136 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db67b77b37b4c228499044749eeeaa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:33.718364 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db67b77b37b4c228499044749eeeaa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:33.719108 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db67b77b37b4c228499044749eeeaa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:33.719747 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db67b77b37b4c228499044749eeeaa3'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:33.737140 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1298/2582] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:39:33 2022] DELETE /identity/v3/users/9db67b77b37b4c228499044749eeeaa3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:33.910962 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:33.923827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae6d2-3a05-4007-9f64-065fdb70423d service swift] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:33.984116 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:33.984882 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:33.985223 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:33.985552 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] PATH_INFO: `/v3/projects/91560178b31a49a09ae7926966d0b77f` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:33.986902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91560178b31a49a09ae7926966d0b77f)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:33.988877 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.037824 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91560178b31a49a09ae7926966d0b77f', 'request_id': 'req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.044220 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91560178b31a49a09ae7926966d0b77f', 'request_id': 'req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.053601 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91560178b31a49a09ae7926966d0b77f', 'request_id': 'req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.064592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] The token cache is being invalidate because project 91560178b31a49a09ae7926966d0b77f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:39:34.064890 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.065572 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.067104 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1285/2583] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:39:33 2022] DELETE /identity/v3/projects/91560178b31a49a09ae7926966d0b77f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:34.071263 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.072928 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:34.087253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda0883-be36-4bd9-9ab4-5a538c36ee67 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.144032 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.144653 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.144923 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.145189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] PATH_INFO: `/v3/users/9cba3e0d96744847b81eb60565541982` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.146331 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cba3e0d96744847b81eb60565541982)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.148291 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.161624 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:34.199812 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cba3e0d96744847b81eb60565541982'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.217066 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cba3e0d96744847b81eb60565541982'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.224596 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cba3e0d96744847b81eb60565541982'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.225302 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cba3e0d96744847b81eb60565541982'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.225973 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-d964816e-9800-4db4-9f1e-caf777fddd6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cba3e0d96744847b81eb60565541982'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.240969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1299/2584] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:39:34 2022] DELETE /identity/v3/users/9cba3e0d96744847b81eb60565541982 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:34.431666 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.433139 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:34.447259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e113-e1ed-488b-a7e7-d0ac0a93d9f1 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.465938 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:34.511019 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.511705 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.511973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.512281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] PATH_INFO: `/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.513474 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=56a4289ba87f4e0f8e7f8959b01cf7f6)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.515569 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.562803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56a4289ba87f4e0f8e7f8959b01cf7f6', 'request_id': 'req-e093acb1-2897-4a41-af4a-59fbec6a14ad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.569109 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56a4289ba87f4e0f8e7f8959b01cf7f6', 'request_id': 'req-e093acb1-2897-4a41-af4a-59fbec6a14ad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.578775 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56a4289ba87f4e0f8e7f8959b01cf7f6', 'request_id': 'req-e093acb1-2897-4a41-af4a-59fbec6a14ad'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.589956 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] The token cache is being invalidate because project 56a4289ba87f4e0f8e7f8959b01cf7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:39:34.590278 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.590910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-e093acb1-2897-4a41-af4a-59fbec6a14ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:39:34.592434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1286/2585] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:39:34 2022] DELETE /identity/v3/projects/56a4289ba87f4e0f8e7f8959b01cf7f6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:34.597676 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.598176 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.598275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.619318 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:34.620548 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:34.681985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1300/2586] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:34.691053 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.691380 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.691655 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.716095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:34.717307 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:34.726604 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1287/2587] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:34.731121 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.741078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafdacd-73f7-472f-87f4-7a81463b4a2a None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.747037 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.747791 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.748104 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.748348 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.749005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81436) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 25 13:39:34.749309 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.750914 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.763608 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1301/2588] 10.222.0.36 () {58 vars in 1153 bytes} [Tue Oct 25 13:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 25 13:39:34.768006 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.777647 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244d69-cc59-4192-b088-b604a27a0e82 None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.783682 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.784280 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.784558 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.784828 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.785461 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.787411 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.818211 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1288/2589] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:39:34 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:39:34.822513 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.832937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60abeb-f5ff-4990-9930-49d38eb49842 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.894274 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.894885 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.895183 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.895448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.896106 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.898025 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.921949 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1302/2590] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:39:34 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:39:34.926309 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.937086 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18db21-8288-44d2-9018-59739b9cfc4c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.941883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.942472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.942746 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.943013 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.943704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.945290 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.955259 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1289/2591] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:39:34.959275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:34.970223 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795e7c5-bec7-48e6-96ba-78045c6149c4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:34.975701 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:34.976292 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:34.976568 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:34.976835 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:34.977483 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:34.979069 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f031bd-533b-446b-95b4-d146439a0a6a admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:34.981172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:34.991440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1303/2592] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:39:34.995112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:35.005291 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c9ae-0c14-4cf6-9173-3b90e12fcbae admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:35.010347 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:35.010964 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:35.011281 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:35.011546 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] PATH_INFO: `/v3/projects/0698776dd0fa4784b387d19fdbd7d4b3/users/2b09ed8449504a2dbd6a18aeb2d10328/roles/4fe0dc8c5db444d78f100f81099859c7` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:35.026561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0698776dd0fa4784b387d19fdbd7d4b3, user_id=2b09ed8449504a2dbd6a18aeb2d10328, role_id=4fe0dc8c5db444d78f100f81099859c7)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:35.029318 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:35.046662 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1290/2593] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:39:34 2022] PUT /identity/v3/projects/0698776dd0fa4784b387d19fdbd7d4b3/users/2b09ed8449504a2dbd6a18aeb2d10328/roles/4fe0dc8c5db444d78f100f81099859c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:35.051941 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b522-39f7-471a-8d06-3ccc3dd63e68 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:35.052234 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b522-39f7-471a-8d06-3ccc3dd63e68 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:35.052505 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b522-39f7-471a-8d06-3ccc3dd63e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:35.071690 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-f363b522-39f7-471a-8d06-3ccc3dd63e68 None None] MFA Rules not processed for user `ec20b39fa45c4429b8c7ce4fb9b19e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:35.072900 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-f363b522-39f7-471a-8d06-3ccc3dd63e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:35.136387 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1304/2594] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:35.146197 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:35.148034 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:35.201387 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:35.208355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6164b0b-a17c-4093-bd2e-71606d909b7c admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:35.213963 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:35.214623 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:35.214904 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:35.215207 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:35.224476 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:35.225909 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:35.235383 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1291/2595] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:39:39.355172 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.355802 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.356005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.386105 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] MFA Rules not processed for user `2b09ed8449504a2dbd6a18aeb2d10328`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:39.387503 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:39.456629 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1305/2596] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:39.461620 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.471799 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2754608-1321-4025-84f8-2c40ad66f080 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.476910 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.477507 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.477785 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.478054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.478698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.480708 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.509533 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1292/2597] 10.222.0.36 () {60 vars in 1153 bytes} [Tue Oct 25 13:39:39 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:39:39.513894 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.526175 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd97e19-a38b-4259-8ce3-df824420b38b None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.586487 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.587124 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.587439 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.587705 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] PATH_INFO: `/v3/users` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.588352 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.590386 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.614717 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1306/2598] 10.222.0.36 () {60 vars in 1144 bytes} [Tue Oct 25 13:39:39 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 25 13:39:39.618671 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.629596 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb6dcb-cd93-4be3-93ae-13a068c5bfd2 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.634475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.635089 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.635399 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.635715 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.636502 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.638274 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.649324 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1293/2599] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 25 13:39:39.653722 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.663684 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b378-f100-4c18-a7f0-cc2987f8251c admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.668510 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.669170 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.669443 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.669724 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] PATH_INFO: `/v3/projects/8bafea19e9584846a772692ab24190dc/users/5cabd640e52544a0b121a3e09963ef84/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.684346 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bafea19e9584846a772692ab24190dc, user_id=5cabd640e52544a0b121a3e09963ef84, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.687005 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.689193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:39.704114 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1307/2600] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:39:39 2022] PUT /identity/v3/projects/8bafea19e9584846a772692ab24190dc/users/5cabd640e52544a0b121a3e09963ef84/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:39.708316 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.719516 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d00a5-9e5e-447c-b487-f5a08c027dc0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.776753 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.777394 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.777674 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.777929 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.778574 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.780177 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b009d2-25b4-4cf9-b2a6-ab0e345251ac admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.790041 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1294/2601] 10.222.0.36 () {58 vars in 1122 bytes} [Tue Oct 25 13:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 25 13:39:39.794492 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:39.805960 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b687951-af43-4f7a-aae7-926cb531ca98 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:39.812130 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:39.812921 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:39.813202 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:39.813473 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] PATH_INFO: `/v3/projects/8bafea19e9584846a772692ab24190dc/users/5cabd640e52544a0b121a3e09963ef84/roles/7ce3b8059d8449b1b4ebed7b09183e21` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:39.815525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bafea19e9584846a772692ab24190dc, user_id=5cabd640e52544a0b121a3e09963ef84, role_id=7ce3b8059d8449b1b4ebed7b09183e21)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:39.818199 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:39.829354 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1308/2602] 10.222.0.36 () {60 vars in 1483 bytes} [Tue Oct 25 13:39:39 2022] PUT /identity/v3/projects/8bafea19e9584846a772692ab24190dc/users/5cabd640e52544a0b121a3e09963ef84/roles/7ce3b8059d8449b1b4ebed7b09183e21 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:39:43.967902 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c3c9-8923-4263-9784-6472552214a2 None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:43.969054 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c3c9-8923-4263-9784-6472552214a2 None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:43.969475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c3c9-8923-4263-9784-6472552214a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:44.026192 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-99b2c3c9-8923-4263-9784-6472552214a2 None None] MFA Rules not processed for user `5cabd640e52544a0b121a3e09963ef84`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:39:44.028004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-99b2c3c9-8923-4263-9784-6472552214a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:44.133582 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1295/2603] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:39:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:39:44.148193 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:39:44.151482 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:39:44.238495 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3e572-4841-43b7-a654-772796e339ee admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:39:44.245672 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:39:44.246647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:39:44.246837 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:39:44.247276 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:39:44.312304 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:39:44.313969 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:39:44.324562 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1309/2604] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:39:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:39:54.127237 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:39:54.319599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:INSERT=2,keystone:SELECT=63 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:40:01.816916 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] REQUEST_METHOD: `POST` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:01.817439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:01.817439 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:01.842231 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.auth.core [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] MFA Rules not processed for user `2b09ed8449504a2dbd6a18aeb2d10328`. Rule list: `[]` (Enabled: `True`). {{(pid=81437) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:40:01.843472 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:01.851051 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1296/2605] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:40:01.895712 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:01.897275 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:01.959626 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34724cdd-b2fc-4f31-a733-5068b694094b service nova] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:01.964374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7623623b296e4d04949973891b697c62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:01.964985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:01.965262 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:01.965537 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:01.967480 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:01.982159 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:01.983665 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:01.995178 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1310/2606] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:02.740958 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:02.742632 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:02.819525 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70a961-4854-48f0-8a5e-198c866e345f None None] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:02.824614 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:02.825218 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:02.825493 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:02.825760 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:02.827529 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:02.839004 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:02.840714 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:02.850435 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1297/2607] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:04.825453 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:04.892265 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f01cd-5a75-42de-b740-bec84dbcd082 service cinder] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:04.896241 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:04.897240 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:04.897516 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:04.897784 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:04.899648 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:04.910252 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:04.911713 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:04.920168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1311/2608] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:05.561370 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:05.562967 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:05.579672 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871b81f-e84d-4449-85c0-e373a3d01e00 service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:05.586073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:05.586684 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:05.586965 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:05.587266 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:05.589017 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:05.602295 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:05.603936 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:05.622121 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1298/2609] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:14.916377 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=52 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:40:15.293970 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:15.309313 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac370d32-c61b-4488-8843-473668860356 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:15.315007 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:15.315637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:15.315917 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:15.316189 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:15.317951 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:15.330474 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:15.332245 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25da7-a379-4a25-b7d6-1849758665db service nova] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:15.343814 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1312/2610] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:15.617839 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=41 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:40:18.416457 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:40:18.418042 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:40:18.431704 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9545f4-dee2-4e44-b327-4fd6d4e5bb8a service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:40:18.436829 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:40:18.437434 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:40:18.437716 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:40:18.437983 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:40:18.448327 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:40:18.449883 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:40:18.459547 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1299/2611] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:40:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:40:25.339402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=6 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:40:28.455524 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:42:22.517205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd3b2-7120-4858-b44b-f78acdf33a8c None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:42:22.517739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd3b2-7120-4858-b44b-f78acdf33a8c None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:42:22.517739 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dd3b2-7120-4858-b44b-f78acdf33a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:42:22.538011 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-d29dd3b2-7120-4858-b44b-f78acdf33a8c None None] MFA Rules not processed for user `2b09ed8449504a2dbd6a18aeb2d10328`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:42:22.539243 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-d29dd3b2-7120-4858-b44b-f78acdf33a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:42:22.547536 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1313/2612] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:42:32.532125 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=3 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:43:22.955505 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:43:22.957112 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:43:23.045422 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56f572-0479-48c6-b5b0-1ee6eca60704 service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:43:23.051000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:43:23.051633 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:43:23.051960 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:43:23.052243 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:43:23.054073 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:43:23.065438 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:43:23.067000 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:43:23.076930 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1300/2613] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:43:33.072673 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=37 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:44:20.568572 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] REQUEST_METHOD: `POST` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:44:20.569150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:44:20.569150 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:44:20.588647 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.auth.core [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] MFA Rules not processed for user `2b09ed8449504a2dbd6a18aeb2d10328`. Rule list: `[]` (Enabled: `True`). {{(pid=81436) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 25 13:44:20.589843 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:44:20.596881 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1314/2614] 10.222.0.36 () {58 vars in 958 bytes} [Tue Oct 25 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 25 13:44:20.924035 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:44:20.943593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9959-3c35-4dbd-b8fd-7731d7613f3f service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:44:20.948039 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:44:20.948639 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:44:20.948915 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:44:20.949206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:44:20.951108 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:44:20.974561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:44:20.975973 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:44:20.984191 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1301/2615] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:44:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:44:21.466522 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:44:21.477033 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac292a-fd92-4213-8ba1-c4a358626102 None None] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:44:21.481490 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:44:21.482045 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:44:21.482321 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:44:21.482637 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:44:21.484602 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:44:21.521627 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:44:21.523078 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:44:21.544374 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1315/2616] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:44:30.980460 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=15 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:44:31.541909 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=21 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:45:26.605784 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:45:26.615314 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63f886-d8f2-48d0-bbb4-c3eabcd82bef service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:45:26.619878 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:45:26.620523 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:45:26.620931 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:45:26.621206 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:45:26.629923 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:45:26.631353 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:45:26.639786 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1302/2617] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:45:36.636099 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=6 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:19.977499 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:19.991883 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4faa2-0e68-43b5-af6c-0cc7a3b0bcd0 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:19.996560 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:19.997166 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:19.997440 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:19.997707 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:20.007127 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:20.008599 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:20.017855 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1316/2618] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:20.898275 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:20.915449 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f145-2f56-444c-b2e4-b83808e9b94a service neutron] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:20.921355 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:20.922095 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:20.922473 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:20.923055 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:20.925001 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:46:20.938970 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:20.940761 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:20.954277 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1303/2619] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:21.070781 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:21.080062 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ea293-00af-4ed9-8e52-bf86ab70b154 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:21.083819 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:21.084706 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:21.084985 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:21.085285 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:21.096357 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:21.097792 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:21.109546 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1317/2620] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:23.915545 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:23.927698 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb975357-3819-4efb-b80e-9902a37ac247 service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:23.934592 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45a976b6672412f92f9a4f8fbc75b1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:23.935251 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:23.935479 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:23.935768 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:23.945809 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:23.947485 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:23.963675 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1304/2621] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:31.105112 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=12 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:33.955625 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=12 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:35.643402 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:35.652749 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9366a-b31d-41f0-b37a-bb113f72fa20 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:35.656995 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd98a9aa326741fba8ff2360f8e87e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:35.657617 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:35.657877 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:35.658188 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:35.732300 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:35.733744 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:35.752905 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1318/2622] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:38.826836 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:38.843579 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4808-8604-462e-b7ee-becf1d91547e service nova] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:38.844382 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:38.845118 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] REQUEST_METHOD: `GET` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:38.845465 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:38.845803 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:38.847750 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:46:38.860106 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:38.861694 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:38.876952 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1305/2623] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:46:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:46:41.858103 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:41.872371 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c0f92-1cf5-4248-8e18-0a550a360268 service neutron] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:41.877718 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:41.878366 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:41.878666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:41.878931 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] PATH_INFO: `/v3/users/2b09ed8449504a2dbd6a18aeb2d10328` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:41.880215 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b09ed8449504a2dbd6a18aeb2d10328)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:41.882253 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:41.932167 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b09ed8449504a2dbd6a18aeb2d10328'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:41.944925 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b09ed8449504a2dbd6a18aeb2d10328'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:41.952534 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b09ed8449504a2dbd6a18aeb2d10328'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:41.953504 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b09ed8449504a2dbd6a18aeb2d10328'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:41.954246 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b09ed8449504a2dbd6a18aeb2d10328'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:41.969703 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1319/2624] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:46:41 2022] DELETE /identity/v3/users/2b09ed8449504a2dbd6a18aeb2d10328 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:46:42.152725 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:42.167594 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e3e6e-50ab-4b0f-91dc-76d0109c9e6d service placement] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:42.231693 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:42.232401 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:42.232593 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:42.232881 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] PATH_INFO: `/v3/projects/0698776dd0fa4784b387d19fdbd7d4b3` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:42.234012 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0698776dd0fa4784b387d19fdbd7d4b3)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:42.236679 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:42.285509 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0698776dd0fa4784b387d19fdbd7d4b3', 'request_id': 'req-b753de74-251d-4468-8157-592f4ad14da0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.291897 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0698776dd0fa4784b387d19fdbd7d4b3', 'request_id': 'req-b753de74-251d-4468-8157-592f4ad14da0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.300354 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0698776dd0fa4784b387d19fdbd7d4b3', 'request_id': 'req-b753de74-251d-4468-8157-592f4ad14da0'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.323918 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] The token cache is being invalidate because project 0698776dd0fa4784b387d19fdbd7d4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:46:42.324196 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.324827 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.326669 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1306/2625] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:46:42 2022] DELETE /identity/v3/projects/0698776dd0fa4784b387d19fdbd7d4b3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:46:42.330761 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:42.332281 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:46:42.346117 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95f93-8bd8-4a39-8118-87f15d3635f4 admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:42.393226 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:42.407424 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:42.408079 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:42.408342 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:42.408609 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] PATH_INFO: `/v3/users/5cabd640e52544a0b121a3e09963ef84` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:42.409779 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cabd640e52544a0b121a3e09963ef84)` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:42.411760 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:42.460448 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cabd640e52544a0b121a3e09963ef84'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.473289 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cabd640e52544a0b121a3e09963ef84'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.480604 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cabd640e52544a0b121a3e09963ef84'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.481310 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cabd640e52544a0b121a3e09963ef84'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.481937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.notifications [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cabd640e52544a0b121a3e09963ef84'} {{(pid=81436) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.498607 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1320/2626] 10.222.0.36 () {58 vars in 1224 bytes} [Tue Oct 25 13:46:42 2022] DELETE /identity/v3/users/5cabd640e52544a0b121a3e09963ef84 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:46:42.678264 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Authenticating user token {{(pid=81437) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:46:42.679802 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.fernet_utils [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81437) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:46:42.692862 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753de74-251d-4468-8157-592f4ad14da0 admin admin] Validating token access rules against request {{(pid=81437) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:46:42.752724 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec20b39fa45c4429b8c7ce4fb9b19e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e7cf0d7d46da4af3b19a1ac43ed2227b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81437) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:46:42.753283 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:46:42.753561 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:46:42.753838 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] PATH_INFO: `/v3/projects/8bafea19e9584846a772692ab24190dc` {{(pid=81437) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:46:42.755016 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bafea19e9584846a772692ab24190dc)` {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:46:42.756991 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] RBAC: Authorization granted {{(pid=81437) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:46:42.764972 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:42.807781 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bafea19e9584846a772692ab24190dc', 'request_id': 'req-fc948484-9825-422e-87eb-08e57401dda2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.814048 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bafea19e9584846a772692ab24190dc', 'request_id': 'req-fc948484-9825-422e-87eb-08e57401dda2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.822719 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bafea19e9584846a772692ab24190dc', 'request_id': 'req-fc948484-9825-422e-87eb-08e57401dda2'} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.833135 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] The token cache is being invalidate because project 8bafea19e9584846a772692ab24190dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81437) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 25 13:46:42.833423 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.834475 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG keystone.notifications [None req-fc948484-9825-422e-87eb-08e57401dda2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81437) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 25 13:46:42.836033 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: [pid: 81437|app: 0|req: 1307/2627] 10.222.0.36 () {58 vars in 1233 bytes} [Tue Oct 25 13:46:42 2022] DELETE /identity/v3/projects/8bafea19e9584846a772692ab24190dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 25 13:46:52.494205 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:46:52.830348 dv-cinder-862432-2-iscsi devstack@keystone.service[81437]: DEBUG dbcounter [-] [81437] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 25 13:48:29.906481 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Authenticating user token {{(pid=81436) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 25 13:48:29.908054 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:48:29.990168 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f2d38-f985-4881-b334-e2677ff2a5fc admin admin] Validating token access rules against request {{(pid=81436) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 25 13:48:30.051324 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8112d8c2ea24bd286a5f139ba765f2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb51413c1144daab7dc47dec8f28c10', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81436) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 25 13:48:30.051937 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] REQUEST_METHOD: `GET` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 25 13:48:30.052217 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] SCRIPT_NAME: `/identity` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 25 13:48:30.052489 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81436) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 25 13:48:30.054454 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.fernet_utils [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81436) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 25 13:48:30.065896 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 25 13:48:30.067525 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448538a6-3a54-46c4-a268-fb4ab04d049f service placement] RBAC: Authorization granted {{(pid=81436) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 25 13:48:30.078216 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: [pid: 81436|app: 0|req: 1321/2628] 10.222.0.36 () {60 vars in 1365 bytes} [Tue Oct 25 13:48:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 25 13:48:40.073666 dv-cinder-862432-2-iscsi devstack@keystone.service[81436]: DEBUG dbcounter [-] [81436] Writing DB stats keystone:SELECT=58 {{(pid=81436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}